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THE  BASIC  RESEARCH  PROGRAM:  IDENTIFYING 
NEW  AND  PROMISING  TECHNOLOGIES 

The  Basic  Research  Unit  (BRU)  research  program  focuses  on  providing  the  basic 
research  underpinnings  for  the  personnel,  leader  development,  and  training  requirements  of  the 
future.  This  program  is  a  critical  link  between  the  world  of  behavioral  science  and  the  military 
community.  Searching  out  and  advancing  the  state-of-the-art  methods,  theories,  and  findings  in 
behavioral  science;  encouraging  projects  most  likely  to  contribute  generalizable  scientific 
principles  and  new  knowledge;  and  supporting  those  efforts  that  have  potential  military 
relevance  and  likelihood  of  leading  to  applied  behavioral  technology  are  BRU’s  key  research 
goals. 


Universities  conduct  most  of  the  basic  research  in  the  program.  BRU  maintains  close 
contact  with  other  basic  behavioral  science  research  organizations  as  well  as  with  ARI's  applied 
researchers,  other  relevant  agencies  within  the  Army,  and  other  military  services.  These  contacts 
help  to  define  issues  that  require  fundamental  research  and  facilitate  the  transition  of  basic 
research  results  to  applied  programs  for  eventual  use  by  the  operational  Army. 

In  BRU's  contract  programs,  a  Broad  Agency  Announcement  (BAA)  is  issued  each  year 
to  solicit  both  concept  papers  and  formal  proposals  relating  to  the  announced  program  research 
areas.  In  a  given  year,  the  BAA  highlights  the  research  objectives  of  special  interest  and  provides 
an  open  call  for  proposals. 

On  the  following  pages,  the  reader  will  find  summaries  of  current  and  recently  completed 
BRU  contracts,  which  began  between  2001  and  2005.  There  are  three  current  BRU  program 
objectives,  each  of  which  is  discussed  in  detail  below.  These  objectives 

1 .  To  provide  fundamental  knowledge  to  improve  training  in  complex  environments; 

2.  To  provide  fundamental  knowledge  to  improve  leader  and  team  performance;  and 

3.  To  provide  fundamental  knowledge  for  identifying,  assessing,  and  assigning  quality 

personnel  in  the  evolving  Army. 

Training  in  Complex  Environments 

Basic  research  in  this  area  focuses  on  developing  concepts  and  methods  for  training 
complex  tasks  and  for  sustaining  complex  task  performance.  Assessing  the  cognitive  impact  of 
technology  requirements  arising  from  digital,  semi -automated,  and  robotic  systems  on  training 
requirements  is  also  part  of  this  process.  One  of  the  efforts  in  this  area  seeks  to  find  conditions 
that  simultaneously  optimize  learning,  memory,  and  transfer  of  training.  We  seek  to  identify 
unique  training  principles  and  methods  for  improving  interpersonal  skills  and  team  adaptability 
and  performance.  The  expected  outcome  is  applied  research  testing  the  principles  and  methods 
produced  in  this  work  package  in  Army  training  environments.  The  models  and  theories 
produced  should  be  useful  in  accounting  for  individual  differences  in  training  and  facilitating 
practical,  individualized,  adaptive  training  methods.  Another  effort,  for  example,  centers  on  the 
characteristics  of  learners  and  the  training  enviomment  that  lead  to  either  facilitative  or 
inhibiting  effects  of  seductive  details  embedded  within  the  training  materials,  while  another 


1 


addresses  team  learning  from  a  distance  under  a  variety  of  contexts.  These  advances  will 
translate  into  improved  training  methods  in  a  wide  range  of  tasks. 

Improving  Leader  and  Team  Performance 

Commensurate  with  the  requirements  for  rapidly  developing  adaptable,  flexible  leaders, 
the  basic  research  program  in  improving  leader  performance  is  directed  toward  providing 
concepts  and  methods  for  accelerating  leader  development  and  understanding  and  developing 
leader  adaptability  and  flexibility  in  a  manner  that  can  be  tested  in  the  applied  environment.  One 
of  our  efforts  is  focused  on  developing  both  adaptive  leaders  and  teams  through  formal 
instruction,  developmental  work  assignments,  and  self-development.  We  are  also  committed  to 
discovering  and  testing  the  basic  cognitive  principles  that  underlie  effective  leader-team 
performance.  Another  effort  examines  predictors  of  adaptive  performance  and  the  development 
of  team  leadership  behaviors  under  the  stress  of  performing  in  a  metropolitan  hospital  shock 
trauma  center  and  in  the  laboratory.  We  anticipate  that  results  from  this  research  will  make  an 
important  contribution  to  understanding  and  improving  organizational  effectiveness. 

Quality  Personnel  for  the  Evolving  Army 

Identifying  and  measuring  the  aptitudes  and  skills  that  are  unique  to  the  human 
performance  requirements  of  military  service  is  a  major  theme  of  this  basic  research  effort.  As 
part  of  this  process,  we  seek  to  devise  methods  that  assess  mental  flexibility  and  adaptability, 
describe  how  these  attributes  develop,  and  measure  their  contribution  to  performance  and  job 
tenure.  For  example,  one  of  our  efforts  seeks  new  ways  to  conceptualize  and  assess  individual 
mental  flexibility  since  mental  flexibility  plays  a  critical  role  in  the  successful  application  of  the 
mental  processes  needed  for  adapting  to,  selecting,  and  shaping  the  environment  in  novel 
situations.  Exploring  the  sociological  and  psychological  factors  that  could  influence  recruitment, 
retention,  and  performance  is  part  of  the  work  in  this  area  as  well.  Another  effort  is  concerned 
with  developing  a  valid,  culture-fair  test  of  intelligence  that  could  be  applied  in  selection  and 
placement.  Applied  research  that  builds  the  understanding  of  the  cognitive  processes  entailed  in 
flexible  thinking  can  be  used  to  identify  those  who  are  flexible,  adaptable  thinkers  and  to  develop 
these  skills  through  effective  instructional  programs  that  facilitate  the  use  of  flexible  thinking 
strategies  in  a  variety  of  contexts. 

This  document  provides  a  listing  and  brief  synopsis  of  ongoing  and  recently  completed 
research  efforts.  Project  listings  are  organized  into  the  three  aforementioned  research  objectives. 

It  is  important  to  note,  however,  that  basic  research  is  but  one  of  many  programs  for  which  BRU 
has  responsibility. 

Other  programs  in  BRU  include: 

•  Small  Business  Innovative  Research  (SBIR)  Program; 

•  Small  Business  Technology  Transfer  (STTR)  Program; 

•  International  Behavioral  Science  and  Technology  Watch; 

•  Graduate  student  apprenticeship  program  -  the  Consortium  Research  Fellows  Program 

with  the  Consortium  of  Metropolitan  Washington  Universities;  and 


•  Outreach  efforts  to  Historically  Black  Colleges  and  Universities  (HBCUs)  and  Minority 
Institutions  (Mis),  and  research  support  in  behavioral  science  for  the  U.S.  Military 
Academy. 

Additional  information  about  reports  from  these  research  efforts  is  available  upon 
request. 


Paul  A.  Gade,  Chief, 

Basic  Research  Unit 

Tiffany  M.  Bludau,  Consortium  Fellow, 
Basic  Research  Unit 
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BRU  RESEARCH  OBJECTIVE  #1:  PROVIDE 
FUNDAMENTAL  KNOWLEDGE  TO  IMPROVE  TRAINING 
IN  COMPLEX  ENVIRONMENTS 


Research  under  this  objective  develops  concepts  and  methods  for 
training  complex  tasks  and  sustaining  complex  task  performance.  The  focus 
is  on  understanding  the  cognitive  impact  of  Future  Force  technology  on 
training  requirements,  the  impact  of  voluminous  multi-modal  data  on 
performance,  and  developing  methods  for  improving  cognitive  skills. 
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Factors  Influencing  the  Design  and  Conduct 
of  Effective  Technology-Delivered  Instruction 


Contract  #:  DASW01-04-K-0002 
Institution:  The  University  of  Tulsa 


Illinois  Institute  of  Technology 


Contract  Dates:  03/01/2004  to  02/28/2006 
PI:  Joanne  Davis  (TU) 

Co-PI:  Annette  Towler  (IIT)  &  Kurt 
Kreiger  (TU) 

Liaisons:  Scott  Graham,  IFRU; 

Jim  Belanich,  BRU 


Problem(s)/Research  Question(s)  -  This  project  examined  the  effect  of  "seductive  details”  on 
training.  Seductive  details  refer  to  entertaining  and  interesting  information  inserted  into  training 
material  that  is  tangential  to  the  main  theme  of  the  topic.  While  previous  research  has  shown  that 
the  use  of  seductive  details  and  on-screen  text  hinders  learning  in  computer-based  training 
environment,  our  research  shows  that  these  factors  at  times  have  no  effect  or  a  positive  effect  on 
learning.  Our  research  centers  on  characteristics  of  learners  and  the  learning  environment  that 
lead  to  either  facilitative  or  inhibiting  effects  of  seductive  details. 

Technical  Barrier(s)  -  Previous  research  establishing  the  seductive  detail  effect  has  used  a 
limited  set  of  training  materials,  has  focused  exclusively  on  declarative  knowledge  and  has  not 
adequately  distinguished  seductive  details  that  are  independent  of  or  dependent  of  the  primary 
training  content.  Little  is  known  about  the  effects  of  prior  knowledge  of  course  content  or 
trainees*  learning  styles.  Finally,  preliminary  research  suggests  that  seductive  details  are  most 
likely  to  have  an  effect  when  learners  approach  their  limits  for  cognitive  load  (how  much 
information  they  can  process  at  once),  and  these  limits  may  vary  from  learner  to  learner. 

Significance/Impact  for  Basic  Research  -  The  results  will  allow  us  to  assess  the  extent  to 
which  training  design  characteristics  impair  both  trainees'  recollection  of  the  facts  and  principles 
taught  in  training  and  their  ability  to  perform  the  skills  that  were  taught.  In  the  long-term,  this 
research  will  enable  the  military  to  design  more  effective  computer-based  training. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Basic  and  applied  research  on  learning  and  instructional  principles; 

•  Instructional  design  for  computer-based  or  Web-based  training;  and 

•  Game-based  training;  determining  whether  details  related  to  game  fidelity  detract  from 
learning  in  game/simulation  environments. 


Overview 


One  common  strategy  for  enhancing  any  type  of  training  is  to  make  the  content  as 
interesting  as  possible.  Intuitively,  these  strategies  should  increase  intrinsic  motivation  in 
learners,  resulting  in  greater  appreciation  for  the  material  and  greater  persistence  in  learning 
environments.  Obviously,  interesting  information  that  is  central  to  the  main  topic  is  useful  and 
can  facilitate  trainees*  learning.  However,  trainers  or  instructional  designers  tend  to  include 
interesting  information  that  is  unrelated  to  the  central  theme  to  spice  up  mundane  information. 
This  phenomenon  is  called  the  "seductive  details  effect’*  and  refers  to  essentially  "highly 
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interesting  and  entertaining  information  that  is  only  tangentially  related  to  the  topic  but  is 
irrelevant  to  the  author’s  intended  theme”  (Harp  &  Mayer,  1998,  p.  1).  Instructional  material  can 
be  seductive  through  inclusion  of  illustrations  or  text  that  is  interesting  and  entertaining  but 
tangential  to  the  topic  (e.g.,  adding  a  picture  of  a  professional  football  team  to  a  training  module 
on  teamwork). 

Although  one  could  argue  that  the  inclusion  of  seductive  details  increases  trainees' 
emotional  interest  in  the  topic,  there  is  evidence  suggesting  that  the  inclusion  of  this  material 
disrupts  trainees’  concentration  on  the  main  themes  of  the  material  (Harp  &  Mayer,  1998; 

Mayer,  Heiser,  &  Lonn,  2001).  Thus,  trainees  learn  less  in  courses  where  seductive  details  are 
included  along  with  the  presentation  of  material  relevant  to  the  topic  of  the  course. 

Previous  seductive  details  research  has  focused  exclusively  on  declarative  knowledge 
(i.e.,  knowledge  of  the  facts  and  principles  taught  in  training),  and  most  studies  have  used  only  a 
single  set  of  instructional  materials  (e.g..  Harp  &  Mayer,  1997;  1998;  Mayer  et  al.,  2001 ).  In 
previous  research  described,  we  found  that  seductive  details  had  no  effect  on  recall  but  had  a 
beneficial  effect  on  transfer. 

Research  Approach  -  Study  1 

Ninety  students  from  two  universities  participated  in  a  10  minute  computer-based 
training  course.  Participants  ranged  from  1 8-29  years  of  age  with  an  average  age  of  2 1  years.  The 
course  demonstrated  several  functions  in  Excel  that  could  be  used  to  select  a  city  to  live  in  based 
on  the  city's  average  rainfall  and  temperature.  Participants  are  randomly  assigned  to  one  of  four 
experimental  conditions;  (1)  training  includes  both  seductive  details  and  on-screen  text,  (2) 
training  includes  seductive  details  but  not  on-screen  text,  (3)  training  includes  on-screen  text  but 
not  seductive  details,  (4)  training  did  not  include  seductive  details  or  on-screen  text.  On-screen 
text  summarizes  key  points  covered  in  training,  while  seductive  details  included  interesting  facts 
about  the  data  used  in  training  such  as  “Tree  crickets  are  called  the  poor  man’s  thermometer 
because  temperature  directly  affects  their  rate  of  activity.  Listen  for  a  cricket  and  count  the 
number  of  chirps  it  makes  in  fifteen  seconds.  Add  37.  The  sum  will  be  the  Fahrenheit 
temperature  (almost  exactly!).” 

Accomplishments  -  Study  1 

Use  of  Excel  prior  to  training  accounted  for  a  significant  8%  of  the  variance  in  procedural 
knowledge  exam  scores.  However,  seductive  details  and  text-summaries  did  not  account  for  a 
significant  portion  of  the  variance  in  declarative  knowledge  (R2  =  .003,  p  >  .05),  procedural 
knowledge  (R2  =  .029,  p  >  .05),  or  training  reactions  (R2  =  .035,  p  >  .05),  after  controlling  for 
previous  experience  with  Excel.  Thus,  including  seductive  details  and/or  text-summaries  did  not 
affect  training  reactions  or  how  much  trainees  learned,  contrary  to  the  hypotheses. 

Seductive  details  and  on-screen  text  interacted  with  trainees'  learning  styles.  Including 
on-screen  text  greatly  decreased  performance  for  trainees  with  an  auditory  learning  style  and 
slightly  decreased  performance  for  trainees  without  an  auditory  learning  style  (see  Figure  1 ). 

This  effect  occurred  for  both  declarative  and  procedural  knowledge  learning  outcomes.  Also, 
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including  seductive  details  greatly  decreased  performance  on  the  procedural  knowledge  exam  for 
trainees  with  a  tactile  learning  style  and  slightly  decreased  performance  for  trainees  without  a 
tactile  learning  style  (see  Figure  2). 


Figure  1.  Interaction  between  text-summaries  and  auditory  learning  style  when  predicting 
declarative  and  procedural  knowledge  test  scores. 


Figure  2.  Interaction  between  seductive  details  and  tactile  learning  style  when  predicting 
procedural  knowledge  test  scores. 


7 


Research  Approach  -  Study  2 

The  second  study  was  a  replication  and  extension  of  the  first  study.  We  attempted  to 
replicate  the  findings  from  Study  1  in  a  different  setting  using  another  training  program 
(Microsoft  Mailmerge).  We  extended  the  second  study  through  testing  the  effects  of  different 
types  of  seductive  details  on  recall  and  transfer  performance.  To  better  understand  the  seductive 
details  effect,  we  tested  whether  they  had  a  deleterious  effect  depending  on  whether  they  were 
independent  or  dependent  on  instructional  content.  For  example,  in  the  context  of  providing 
instruction  on  the  use  of  Microsoft  Mailmerge,  an  example  of  dependent  seductive  detail  is  “‘The 
Windows  NT  software  was  developed  by  Microsoft-lsrael,"  whereas  an  independent  seductive 
detail  is  “Top  graduate  schools  often  accept  up  to  only  10  applicants  per  year.*'  As  can  be  seen  in 
the  examples,  dependent  seductive  details  are  tangentially  related  to  the  instructional  material, 
whereas  independent  seductive  details  have  no  relevance  to  the  instructional  material. 

Seventy-seven  participants  were  recruited  for  this  study  from  a  private  Midwestern 
university.  The  independent  variable  consisted  of  three  levels:  no  seductive  details,  independent 
seductive  details,  and  dependent  seductive  details.  As  in  Study  1,  post-training  measures  were 
collected  on  a  multiple-choice  test  of  declarative  knowledge  and  a  performance-based  measure 
of  transfer  after  a  time  delay  and  distracter  task. 

Accomplishments  -  Study  2 

As  in  Study  1,  there  was  no  main  effect  for  seductive  detail  (SD)  condition  on  declarative 
knowledge  test  scores.  There  was,  however,  a  main  effect  for  seductive  detail  condition  on 
transfer  performance.  Transfer  performance  was  better  in  groups  that  were  exposed  to 
independent  seductive  details  for  the  transfer  task  score  (A/=  15.62,  SD  =  1 .03)  compared  to  the 
control  group  {M  =  11.15,  SD  =  1.17).  There  was  no  difference  between  the  dependent  seductive 
details  (M  =  13.03,  SD  =  1.07)  and  the  other  groups.  Consistent  with  study  1  (and  contrary  to  the 
findings  of  many  previous  studies),  the  introduction  of  (independent)  seductive  details  had  a 
facilitative  effect  on  transfer  performance. 

Research  Approach  -  Study  3 

Since  we  conducted  two  studies  with  results  that  were  very  different  than  those  obtained 
by  Mayer  and  colleagues  (e.g..  Harp  &  Mayer,  1997;  1998;  Mayer  et  al„  2001 ),  we  attempted  to 
conduct  a  tight  replication  of  one  of  Mayer's  earlier  studies.  Because  results  of  both  prior  studies 
conflict  with  those  of  previous  research,  we  conducted  a  third  investigation  attempting  to 
replicate  as  closely  as  possible  conditions  used  in  prior  seductive  details  studies.  Sixty-two 
undergraduates  at  a  Southwestern  university  participated  in  the  investigation  for  extra  credit. 
Following  the  design  in  Harp  and  Mayer  (1998),  participants  were  assigned  to  one  of  four 
conditions:  (1)  a  baseline  condition,  (2)  baseline  plus  seductive  details,  (3)  baseline  plus 
seductive  illustrations;  and  (4)  baseline  plus  seductive  illustrations  and  seductive  text.  In  all 
conditions,  participants  read  a  scientific  explanation  of  how  lightening  forms.  Seductive  details 
were  in  the  form  of  interesting  information  about  lightening  but  irrelevant  to  its  formation  (e.g., 
the  number  of  Americans  per  year  struck  by  lightening)  and/or  interesting  pictures  irrelevant  to 
lightening  formation  (e.g.,  a  high  school  football  player  whose  uniform  had  been  burned  by 
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lighting).  All  text  and  seductive  details  were  identical  to  those  used  by  Harp  and  Mayer. 
Participants  completed  the  training  material  and  answered  four  open-ended  questions  applying 
what  they  had  learned.  Participant  responses  were  scored  by  two  researchers  and  summed  across 
questions.  Note  that  these  transfer  questions  are  actually  problem-solving  or  applied  questions 
rather  than  performance-based  measures  as  used  in  Studies  1  and  2. 

Accomplishments  -  Study  3 

Two  one-way  ANOVA  revealed  significant  main  effects  for  seductive  details  on  both  the 
number  of  main  points  recalled  and  scores  across  the  four  •‘transfer”  or  problem-solving 
questions.  Post  hoc  comparisons  of  recall  measures  indicate  that  participants  recalled  the  most 
main  points  in  the  base  (no  seductive  detail)  condition,  and  the  fewest  main  points  in  the  base  + 
text  +  illustration  condition.  Post  hoc  comparisons  revealed  that  problem-solving  scores  were 
lowest  in  the  base  +  text  +  illustration  condition,  but  there  were  no  differences  between  the  other 
three  conditions.  In  general,  these  results  are  similar  to  those  obtained  by  Harp  and  Mayer 
(1998),  although  not  as  strong  for  the  seductive  detail  text  and  seductive  detail  illustration 
conditions.  Interestingly,  Harp  and  Mayer  concluded  that  ‘'seductive  details  do  their  damage”  by 
interfering  with  the  organization  of  information  at  the  point  of  learning.  However,  visual 
inspection  of  the  free  recall  measures  in  the  three  seductive  details  conditions  shows  strong 
organization  of  information.  Even  though  upon  presentation  the  seductive  details  were 
interspersed  with  the  primary  information,  at  recall,  participants  frequently  presented  the 
seductive  details  (organized)  together  at  either  the  beginning  or  end  of  their  recall  statements. 

Research  Approach  -  Study  4 

In  a  phone  conversation  with  Richard  Mayer,  Dr.  Mayer  speculated  that  one  difference 
between  some  of  our  studies  and  his  was  that  our  studies  provided  less  cognitive  load  on 
participants,  and  thus  seductive  details  were  less  disruptive.  For  example,  when  learning  new 
software,  there  may  be  a  natural  organization  to  the  information,  or  learners  can  rely  on  prior 
knowledge  of  other  software  to  organize  the  new  information.  Seductive  details  could  be  less 
disruptive  under  these  conditions  than  if  all  information  was  new.  Seductive  details  in  general 
would  be  predicted  to  have  the  greatest  effect  when  greater  cognitive  load  was  greatest.  Dr. 
Mayer  also  speculated  that  there  could  be  differences  in  the  nature  of  our  seductive  details  (from 
his).  Over  time,  he  had  found  that  visually  graphic  or  violent  details  tended  to  be  the  most 
disruptive. 

To  examine  the  effects  of  different  levels  of  cognitive  load  and  different  types  of 
seductive  details,  we  conducted  a  fourth  study.  Sixty  research  participants  from  a  private 
southwest  university  were  recruited  for  the  study.  They  were  assigned  to  one  of  four  cells  in  a 
2x2  design.  Participants  read  about  how  a  car  engine  works.  Participants  either  read  one  page  at  a 
time  or  two  pages  at  a  time  (with  the  latter  condition  requiring  them  to  store  more  information  at 
one  time,  thus  imposing  greater  cognitive  load).  Participants  either  read  and  saw  neutral 
seductive  details,  or  graphic  ones  (e.g.,  a  car  from  a  violent  automobile  accident,  or  female 
models  posing  at  a  new  car  show).  We  used  a  multi-item  declarative  knowledge  measure  to 
assess  learning,  and  participants  also  rated  their  satisfaction  with  the  training  materials. 
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Accomplishments  —  Study  4 

There  was  no  effect  for  cognitive  load  (one  v.  two  pages)  or  seductive  detail  type  (neutral 
v.  graphic)  on  the  knowledge  test  scores.  Participants  reading  two  pages  at  a  time  rated  the 
materials  significantly  more  confusing  than  did  participants  reading  one  page  at  a  time.  In  sum, 
there  was  little  support  for  either  manipulation  on  learning,  although  the  requirement  to  read  two 
pages  at  a  time  may  have  created  more  cognitive  load  for  learners. 

Research  Approach  -  Study  5 

Seventy-one  participants  were  recruited  for  this  study  from  a  private  Midwestern 
university.  The  students  ranged  from  freshman  to  undergraduate  seniors.  Twenty-seven 
participants  were  females  and  44  were  males.  Nearly  all  participants  stated  they  were  not 
familiar  with  Microsoft  Mailmerge.  Each  participant  was  randomly  assigned  to  one  of  four 
groups:  (1)  cognitive  load  +  no  SDs,  (2)  the  control-no  cognitive  load  +  no  SDs,  (3)  cognitive 
load  +  SDs,  and  (4)  no  cognitive  load+  SDs. 

The  training  provided  instruction  to  participants  on  Microsoft  Mailmerge-a  function  of 
the  popular  software  package,  Microsoft  Word.  The  training  session  was  an  audio-visual  file  that 
participants  listened  to  with  headphones  and  viewed  on  a  17-inch  computer  monitor.  The  file 
contained  animated  videos  with  screen  shots  of  Mailmerge  and  Excel  programs.  The  overall 
topic  of  the  training  was  to  use  Mailmerge  to  organize  and  send  personalized  letters  in  mass 
mailings.  The  training  covered  managing  database  information,  filtering  and  sorting  information, 
and  inserting  conditional  statements.  Cognitive  load  was  introduced  through  use  of  a  clicker.  In 
the  cognitive  load  condition,  participants  were  required  to  click  a  clicker  each  time  they  heard 
the  word  “document"  during  training. 

Demographics  were  taken  before  the  training.  Motivation  was  measured  through  an  8- 
item  survey  with  questions  like  “I  am  motivated  to  learn  the  skills  emphasized  in  the  training 
program.”  A  7-item  reaction  measure  included  questions  like  'i  found  the  training  coherent.”  A 
13-question,  multiple  choice  post-test  was  administered  to  measure  declarative  knowledge. 
Transfer  of  training  was  measured  through  a  task  that  included  filtering,  sorting  and  merging  of 
letters. 

Accomplishments  -  Study  5 

An  ANCOVA  was  conducted  to  test  for  an  interaction  between  cognitive  load  and 
seductive  details  with  relation  to  transfer  task  scores.  There  was  a  significant  interaction  between 
cognitive  load  and  seductive  details  (p<  .05  ).  An  ANOVA  was  then  conducted  to  further  test  for 
group  differences  in  training  outcomes.  A  post  hoc  test  (Tukey  HSD)  revealed  that  participants 
who  experienced  cognitive  load  +  SDs,  scored  significantly  (p=.04)  lower  on  transfer  than 
participants  with  no  cognitive  load  +  no  SDs. 
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Contributions  to  Basic  Science 


The  results  from  Studies  1  and  2  suggest  that  seductive  details  can  be  effective  for 
transfer.  Prior  research  has  focused  primarily  on  the  effects  of  seductive  detail  on  recall,  finding 
a  negative  effect.  It  is  interesting  to  learn  that  seductive  details  can  have  a  positive  effect  on  task 
performance  following  a  delay  from  training. 

Results  across  all  studies  suggest  that  seductive  details  may  have  an  effect  on  recall 
measures,  but  these  effects  do  not  occur  when  recognition  (multiple-choice  tests)  are  used. 
Researchers  need  to  exercise  care  in  choosing  the  appropriate  dependent  variable,  and  theories 
on  seductive  details  should  account  for  differential  effects  based  on  test  type. 

Visual  inspection  of  participant  responses  in  Study  3  suggests  that  if  seductive  details 
have  negative  effects  on  learning,  they  do  not  do  so  by  preventing  organization  of  new 
information.  Evidence  from  Study  5  (but  not  Study  4)  suggests  that  increasing  cognitive  load 
may  make  seductive  details  more  disruptive  to  learning. 

Cognitive  load  theory  suggests  that  learners  can  absorb  a  limited  amount  of  material  into 
working  memory  and  if  they  are  overloaded  with  information  then  they  fail  to  form  knowledge 
schemas  that  can  be  transmitted  to  long-term  memory  (Sweller,  1994).  Cognitive  load  may  be 
influenced  by  learner  characteristics  (e.g.,  cognitive  ability  or  learning  style),  depth  and 
complexity  of  content,  or  instructional  design  characteristics.  There  may  be  crucial  •‘tipping 
points”  for  cognitive  load  to  occur  and  thus  for  seductive  details  to  do  their  damage.  This  point 
may  also  be  dependent  on  individual  differences,  which  we  did  not  investigate  in  our  research. 

Potential  Army/Military  Applications 

The  knowledge  acquired  from  this  basic  research  emphasizes  the  importance  of 
monitoring  and  controlling  levels  of  cognitive  load  in  computer-based  training  studies.  Cognitive 
load  thus  emerges  as  a  key  variable  for  understanding  relations  among  learner  capabilities, 
instructional  design  features,  and  learning  during  training.  Seductive  details  are  but  one 
mechanism  for  increasing  cognitive  overload.  A  logical  next  step  is  to  study  more  broadly 
multiple  instructional  strategies  that  enhance  learning  and  transfer  by  decreasing  the  learner's 
cognitive  load.  This  research  can  be  followed  up  by  testing  various  design  principles  rooted  in 
cognitive  psychology  in  terms  of  their  impact  on  cognitive  load  in  various  forms  of  computer- 
based  training  of  work-related  skills. 

Understanding  variables  that  affect  learners'  ability  to  separate  understanding  of 
seductive  details  from  their  understanding  of  core  content  may  have  other  applications  for  the 
military  as  well.  For  example,  in  game-based  training,  seductive  details  take  the  form  of  graphics 
and  special  effects  that  enhance  the  appeal  of  the  game  but  might  be  unrelated  to  the  purpose  of 
training.  Seductive  details  and  cognitive  load  theory  may  provide  a  platform  for  studying 
boundaries  of  game  fidelity.  Alternatively,  identifying  learner  factors  related  to  the  capacity  to 
“separate  wheat  from  chaff’  might  have  implications  for  the  designing  effective  training  of 
intelligence  analysts. 
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Problem(s)/Research  Question(s)  -  This  project  aims  to  test  and  develop  principles  of  training 
that  promote  efficient  learning,  durable  memory,  and  flexible  transfer  performance. 

Technical  Barrier(s)  -  Learning,  memory,  and  transfer  performance  are  not  always  highly 
positively  correlated;  for  example,  rapid  learning  often  leads  to  weak  long-term  retention.  There 
is,  thus,  a  technical  barrier  for  finding  conditions  that  simultaneously  optimize  all  three  aspects 
of  training. 

Significance/Impact  for  Basic  Research  -  The  experiments  conducted  as  a  part  of  this  project 
should  produce  data  that  will  support  the  development  of  a  general  theory  of  training 
encompassing  learning  efficiency,  memory  durability,  and  transfer  flexibility. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in 
the  following  ARI  applied  programs  (among  others): 

•  Virtual  Intelligent  Training  for  Objective  Force  Warriors  (WP  294); 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  (WP 

211); 

•  Future  Force  Warrior  Training  (WP  215); 

•  VICTOR:  Virtual  Individual  and  Collective  Training  for  Objective  Force  Warrior  (WP 
233); 

•  Training  for  Interactive  Distributed  Environments  (WP  214);  and 

•  Simulation-focused  Collective  Aircrew  Training  ( WP  23 1 ). 


Overview 


We  have  developed  a  set  of  training  principles  that  optimize  the  efficiency  and  durability 
of  trained  performance.  In  addition,  we  have  discovered  that  conditions  that  lead  to  durability 
often,  and  perhaps  always,  lead  to  limited  flexibility  or  adaptivity.  In  fact,  we  have  found  that 
training  has  little  or  no  benefit  if  there  are  discemable  differences  between  the  training  and 
testing  situations  in  the  background  or  context,  even  if  there  are  no  changes  made  in  the  primary 
task  requirements.  The  focus  of  the  current  project  is,  thus,  to  develop  training  procedures  for 
knowledge  and  skill  that  will  survive  primary  task  or  background  changes  and,  thereby,  produce 
flexible,  as  well  as  efficient  and  durable,  performance  in  military  tasks. 
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Research  Approach 

The  experiments  in  this  project  are  divided  into  three  major  groups.  Experiments  in  the 
first  group  are  designed  to  understand  how  individuals  can  be  trained  to  contend  with  an 
unpredictable  flow  of  information  often  large  in  quantity,  rapidly  presented,  and  ambiguous. 
Experiments  in  the  second  group  are  aimed  to  identify  training  factors  that  promote  adaptive  and 
flexible  performance  in  the  field.  The  final  group  of  experiments  examines  performance  in 
dynamic  and  changing  task  environments.  We  conclude  with  an  effort  to  create,  still  in  the 
laboratory,  a  complex  set  of  tasks  similar  to  those  encountered  by  a  digitally  proficient  pilot 
operating  a  fully  computerized  cockpit,  and  not  unlike  those  of  the  digitally  proficient  “land 
warrior”  Soldier  of  tomorrow  .  The  major  aim  of  this  set  of  experiments  is  to  determine  the  extent 
to  which  training  principles,  first  established  in  simpler  laboratory  tasks,  generalize  to 
performance  under  these  more  complex  conditions. 

Accomplishments 

In  this  summary,  we  focus  on  accomplishments  in  two  experiments.  Both  of  these 
experiments  relate  to  understanding  how  individuals  can  be  trained  to  contend  with  an 
unpredictable  flow  of  information  often  large  in  quantity,  rapidly  presented,  and  ambiguous.  The 
first  experiment  is  also  relevant  to  the  identification  of  training  factors  that  promote  adaptive  and 
flexible  performance  in  the  field.  These  two  experiments  have  helped  us  to  test  and  to  increase 
the  sphere  of  previously  proposed  training  principles. 

The  first  experiment  used  a  message  comprehension  paradigm  in  which  we  examined 
subjects’  ability  to  remember  and  follow'  navigation  instructions.  We  used  this  task  to  investigate 
variations  in  training  to  determine  ways  to  optimize  the  understanding  of,  memory  for,  and 
execution  of  navigation  instructions.  In  particular,  this  study  tested  three  different  training 
principles  that  had  been  proposed  in  earlier  research  using  different  tasks:  (a)  difficulty  of 
training  (any  condition  that  causes  difficulty  during  learning  may  facilitate  later  retention  and 
transfer),  (b)  specificity  of  training  (retention  and  transfer  are  depressed  when  conditions  of 
learning  differ  from  those  during  subsequent  testing),  and  (c)  variability  of  practice  (variable 
practice  conditions  typically  yield  larger  transfer  effects  compared  with  constant  practice 
conditions). 

To  test  the  relative  strength  of  these  principles,  subjects  were  trained  in  one  of  three 
conditions.  During  training  trials  in  the  easy  condition  subjects  were  given  only  short  messages 
including  from  one  to  three  commands,  in  the  difficult  condition  they  were  given  only  long 
messages  including  from  four  to  six  commands,  and  in  the  mixed  condition  they  were  given 
messages  of  all  six  lengths.  Testing  occurred  following  a  short  delay  filled  with  an  irrelevant 
experiment.  At  testing,  all  subjects  received  messages  of  all  six  lengths.  On  the  basis  of  the 
difficulty  of  training  principle,  subjects  should  do  better  at  testing  overall  with  hard  training  than 
with  easy  training.  On  the  basis  of  the  specificity  of  training  principle,  subjects  should  do  better 
at  testing  on  short  message  lengths  with  easy  training  than  with  hard  training  but  should  do  better 
at  test  on  long  message  lengths  with  hard  training  than  with  easy  training.  That  is,  there  should 
be  an  interaction  between  training  condition  and  message  length.  On  the  basis  of  the  variability 
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of  practice  principle,  subjects  should  do  better  at  test  with  mixed  training  than  with  either  easy  or 
hard  training. 

We  found  a  significant  interaction  of  condition  and  message  length,  as  depicted  in  Figure 
1 .  The  mixed  group  was  best  on  all  lengths  but  length  2,  and  the  easy  group  was  better  than  the 
hard  group  on  the  easy  lengths  (1-3,  which  they  trained  on),  whereas  the  hard  group  was  better 
than  the  easy  group  on  the  hard  lengths  (4-6,  which  they  trained  on).  The  results  support  the 
advantages  of  both  specificity  and  variability  of  training  but  do  not  support  the  hypothesis  that 
difficult  training  would  lead  to  overall  best  performance  at  test.  Thus,  in  terms  of  our  principles, 
we  found  support  for  the  principles  of  specificity  of  training  and  variability  of  practice  but  no 
support  for  the  principle  of  difficulty  of  training. 


Figure  1.  Testing  performance  across  conditions. 

The  second  experiment  continues  a  line  of  investigation  that  we  have  been  pursuing  in 
which  we  are  attempting  to  understand  the  processes  underlying  the  bow-shaped  serial  position 
function  evident  in  studies  of  memory  for  order  information.  Earlier  experiments  in  this  series 
examined  memory  for  information  either  learned  outside  the  laboratory  or  learned  in  a  single  list 
presentation  in  the  laboratory.  In  this  experiment,  we  used  a  novel  training  procedure  in  which 
list  items  were  presented  multiple  times.  This  procedure  enabled  us  to  compare  the  relative 
contributions  of  two  variables  postulated  to  affect  the  serial  position  functions:  familiarity  of  the 
items  and  distinctiveness  of  the  positions.  Another  novel  aspect  of  the  present  study  was  that  it 
tested  memory  for  spatial  order,  rather  than  memory  for  temporal  sequence.  Thus,  unlike  earlier 
studies,  position  distinctiveness  was  not  confounded  with  temporal  distinctiveness. 

Many  military  tasks  require  the  participants  to  learn  a  spatial  ordering  that  changes 
dynamically  in  time.  For  example,  command  personnel  must  learn  where  own  troops,  enemy 
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troops,  and  various  combat  resources  are  located  on  the  battlefield.  Furthermore,  our 
manipulation  of  item  familiarity  was  much  stronger  than  the  manipulations  used  in  previous 
studies  of  the  serial  position  function. 

We  employed  the  technique  we  had  developed  earlier  to  investigate  the  strategic  use  of 
knowledge  principle  (learning  and  memory  are  facilitated  whenever  pre-existing  knowledge  can 
be  employed  as  a  mediator  in  the  process  of  acquisition),  in  which  subjects  listed  10  of  their 
friends  and  relatives,  and  these  names  of  familiar  individuals  were  included  as  stimuli  in  the 
experiment  along  with  1 0  names  of  unfamiliar  individuals.  There  were  two  familiarity  order 
groups  that  differed  in  terms  of  the  assignment  of  names  to  list  positions.  In  the  familiar  first 
group,  familiar  names  were  assigned  to  the  first  two  list  positions,  unfamiliar  names  were 
assigned  to  the  next  two  list  positions,  and  so  on  throughout  the  list  of  20  names.  The  opposite 
assignment  of  names  to  list  positions  was  used  for  the  unfamiliar  first  group.  Following  other 
procedures  we  developed  earlier,  subjects  reconstructed  the  order  of  only  a  12-name  subset  of 
the  full  list  of  20  names.  There  were  three  serial  position  conditions  that  differed  in  terms  of  the 
12-name  subset  of  positions  used  for  order  reconstruction:  1-12,  5-16,  and  9-20. 

As  shown  in  Figures  2  and  3,  there  were  striking  effects  of  both  familiarity  order  group 
and  serial  position  condition  on  the  shape  of  the  serial  position  function,  indicating  that 
familiarity  of  the  items  and  distinctiveness  of  the  absolute  list  positions  both  contribute  to  the 
serial  position  function  in  reconstructing  spatial  order  information.  The  effects  of  item  familiarity 
were  particularly  striking,  thereby  also  extending  the  strategic  use  of  knowledge  principle  to 
tasks  involving  reconstruction  of  order.  This  study  demonstrates  clearly  that  prior  knowledge  can 
be  used  effectively  to  learn  new  position  information,  and  that  finding  should  be  relevant  to 
training  battlefield  commanders  who  are  required  to  leam  location  information,  as  described 
above. 
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Figure  2.  Effects  of  serial  position. 


Figure  3.  Effects  of  serial  position. 

Contributions  to  Basic  Science 

We  have  proposed  theoretical  principles  for  optimizing  training.  The  primary  purpose  of 
our  experiments  is  to  collect  data  that  allow  us  to  accept  or  reject  each  of  these  training 
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principles  that  we  formulated  on  the  basis  of  our  basic  laboratory  research  and  to  develop  and 
test  new  principles  that  are  supported  by  these  data.  For  example,  our  recent  study  of 
comprehension  of  navigation  instructions  allowed  us  to  confirm  the  specificity  of  training  and 
variability  of  practice  principles,  and  our  recent  study  of  serial  order  reconstruction  allowed  us  to 
extend  the  strategic  use  of  knowledge  principle  to  a  new  domain. 

Potential  Army/Military  Applications 

Our  experiments  should  be  used  to  drive  applied  research.  To  illustrate  this  potential 
symbiosis  between  basic  and  applied  research,  we  give  two  brief  examples.  First,  our  research 
has  demonstrated  a  high  degree  of  specificity  from  training  to  subsequent  application.  In  fact,  we 
have  shown  that  training  is  specific  even  to  the  length  of  messages  that  need  to  be  understood 
and  executed.  Test  performance  was  best  following  training  with  all  possible  message  lengths. 
These  findings  have  crucial  implications  for  military  training  because  instructors  may  assume 
that  teaching  a  particular  task  through  a  limited  number  of  examples  will  generalize  fully  to  an 
entire  domain  even  when  the  examples  differ  in  a  fundamental  respect  (e.g.,  length)  from  the  test 
situations.  Flowever,  our  findings  imply  that  to  be  effective,  training  should  incorporate  a  full 
range  of  examples  on  critical  task  dimensions.  In  our  research,  we  are  testing  the  boundary 
conditions  of  this  specificity  of  training  and  variability  of  practice  principles  using  a  variety  of 
laboratory  tasks,  and  we  are  exploring  additional  ways  to  overcome  training  specificity  beyond 
increasing  practice  variability.  Although  the  tasks  used  in  our  research  are  often  components  of 
military  tasks  and  require  digital  proficiency,  they  are  not  the  real  military  tasks  currently  being 
trained  in  the  Army.  We  hope  that  applied  research  units  are  interested  in  testing  whether  the 
training  principles  we  have  developed  would  apply  to  such  real  tasks  and  whether  the  methods 
we  hope  to  develop  for  overcoming  the  problem  of  training  specificity  could  be  adapted  to 
improve  military  training.  To  that  end,  we  visited  and  have  had  discussions  with  Stephen 
Goldberg  (Chief,  Simulator  Systems  Research  Unit)  about  future  collaborative  research  on  the 
applicability  of  our  training  principles  to  simulator  training.  Second,  we  have  shown  that  prior 
knowledge  can  be  used  to  enhance  the  learning  of  spatial  position  information.  We  are  presently 
conducting  research  to  explore  the  boundaries  of  effective  prior  knowledge  utilization.  But  the 
use  of  prior  knowledge  needs  to  be  examined  in  the  context  of  real  military  tasks  as  well,  and  we 
hope  that  applied  research  units  will  be  w  illing  and  able  to  take  on  this  examination. 

Future  Plans 

The  original  contract  had  a  termination  date  of  September  30,  2006.  We  requested  and 
were  granted  a  1-year  no-cost  extension.  During  the  remaining  months  of  the  extension,  we  will 
be  completing  the  analysis  and  write-up  of  the  experiments  we  have  conducted,  and  we  will  be 
submitting  manuscripts,  where  appropriate,  to  professional  journals  in  which  we  will  summarize 
our  findings. 
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Understanding  Aspects  of  Individual  and  Collaborative  Skill  Acquisition  in 
Face-to-Face  and  Distance  Training  Situations 


Contract  #:  DASW01-01-K0003 
Institution:  New  Mexico  State  University 


Contract  Dates:  09/01/2001  to  12/2005 

PI:  Adrienne  Y.  Lee 

Co-PIs:  Douglas  Gillan  &  Nancy  Cooke 


Problem(s)/Research  Question(s)  -  How  do  teams  learn  at  a  distance?  How  does  changing  the 
context  between  training  and  test  affect  performance?  What  kind  of  training  can  we  provide  to 
improve  that  transfer  between  contexts? 

Technical  Barrier(s)  -  Cognitive  issues  for  teams/group  training  is  a  relatively  recent  topic  of 
study.  Teams  of  more  than  two  individuals  are  difficult  to  acquire  and  sustain  for  multi-day 
experiments.  Improved  technology  allows  for  both  more  sophisticated  studies  and  better 
measurement  tools;  however,  this  also  means  relying  on  outside  vendors  (to  supply  the  advance 
software  and  programming  support)  and  also  on-site  technical  support. 

Significance/Impact  for  Basic  Research  -  Through  this  research,  we  will  have  a  better 
understanding  of  the  factors  that  affect  team  learning  across  contexts  (i.e.,  the  effect  of 
interacting  face-to-face  versus  communicating  via  technology,  and  what  effect  the  context  of 
complex  software  has  on  this  interaction). 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Training  for  Interactive  Distributed  Environments;  and 

•  Other  training  programs  such  as  Managing  Training  and  Providing  Feedback  within 
Networked  Embedded  Training. 


Overview 


The  purpose  of  this  research  is  to  examine:  a)  whether  teams  training  together  in  the  same 
location  (co-located)  will  train  more  quickly  and  efficiently  than  teams  who  are  not  in  the  same 
location  (distributed);  b)  whether  testing  in  the  same  situation/context  (co-located  to  co-located 
and  distributed  to  distributed)  is  better  than  training  in  different  situations/contexts  (co-located  to 
distributed  and  distributed  to  co-located);  c)  whether  we  can  develop  training  to  improve  the 
transfer  between  contexts;  and  d)  whether  the  training  will  improve  learning. 

Research  Approach 

This  investigation  asked  teams  of  three  individuals  to  learn  a  complex  task  and  then  use 
that  knowledge  to  work  together  to  solve  some  new  tasks.  Originally,  for  the  tasks,  individuals 
were  asked  to  learn  how  to  operate  an  Unmanned  Air  Vehicle  (UAV)  where  the  three 
individuals,  each  with  different  jobs,  must  fly  a  UAV  to  various  locations  and  take  pictures  of 
specific  targets.  After  an  initial  investigation,  the  new  focus  shifted  to  peacekeeping  missions 
where  three  individuals  manage  different  resources  in  Support  and  Stability  Operations  (SASO). 
Both  of  these  contexts  are  simulated  in  the  laboratory  and  training/testing  is  performed  using  the 
following  methodology. 
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First,  individuals  learn  the  complex  task  either  together  (face-to-face,  co-located)  or  apart 
(distance,  distributed).  Then  a  new,  equally  complex  task  is  chosen  and  the  individuals  are  asked 
to  perform  the  new  task.  They  are  either  asked  to  stay  in  the  same  context  or  change  to  the  other 
context.  Initially,  for  the  UAV  task,  co-located  context,  the  computer  consoles  were  in  a  semi¬ 
circle  that  allowed  individuals  to  speak  over-the-shoulder  to  each  other;  for  the  distance  context, 
the  individuals  were  separated  by  a  screen,  with  one  in  another  room.  For  the  SASO  task,  co¬ 
located,  individuals  are  facing  computes  in  a  circular  configuration  which  allows  them  to  talk  to 
each  other  easily;  for  the  distance  context,  the  individuals  are  in  completely  separate  rooms  with 
sound  material  on  the  walls.  The  new  technology  allows  for  smaller,  lighter  computer  equipment 
that  can  be  easily  moved  to  different  rooms  and  communication  occurs  through  light  headsets. 

Accomplishments 

In  the  spring  and  fall  2002,  the  first  investigation  on  co-located/distance  team  training 
and  transfer  was  performed  in  the  UAV  CERTT  lab.  Performance  results  are  shown  below. 
Although  all  teams  received  the  same  initial  (declarative)  knowledge  training,  distance  teams 
outperformed  co-located  teams  throughout  the  initial  hands-on  training  sessions  and  after 
transfer.  The  distance  teams  reached  a  plateau  but  this  was  probably  due  to  a  lack  of  feedback. 
After  transfer,  all  teams  experienced  a  decrement  in  performance  (similar  to  Singley  & 

Anderson,  1989)  but  the  distance  teams  (i.e.,  those  who  trained  at  a  distance  and  either  stayed  in 
the  same  distance  condition  or  transferred  to  a  co-located  condition)  showed  a  greater 
improvement  for  the  two  training  missions  after  transfer.  These  results  may  have  been  due  to  the 
small  numbers  of  teams  or  the  experimental  setup  in  the  UAV  lab  which  may  have  suppressed 
co-located  performance. 


Missions 


Figure  1.  Performance  across  teams  from  Experiment  1 . 

Note.  A  change  in  context  occurs  at  Mission  7.  CC=  co-located;  CD  =  co-located  to  distributed;  DC  = 
distributed  to  co-located;  DD=  distributed. 
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In  2003  we  developed  the  SASO  lab  and  tested  the  software  both  among  the  graduate 
student  assistants  and  with  practice  teams.  We  were  able  to  get  necessary  changes  to  the  SASO 
software  and  to  debug  that  software,  install  video  and  audio  recording,  re-configure  a  lab  space 
to  house  the  hardware,  and  train  assistants  on  the  task.  From  spring  2004  through  spring  2005, 
we  replicated  the  investigation  in  the  SASO  lab.  The  main  differences  in  physical  context  were 
described  above;  however,  we  added  a  communication  training  component.  Thus,  we  can  see 
whether  the  change  in  the  technology  has  a  bigger  effect  than  a  change  in  the  physical  context. 

We  ran  60  teams  to  test  both  the  conditions  set  forth  for  Experiment  1  and  push-to-talk 
versus  no  push-to-talk.  Results  for  Experiment  2  are  shown  in  Figures  2  through  4.  The  pattern 
of  results  for  push-to-talk  is  different  than  for  no  push-to-talk.  If  you  look  at  the  training  results, 
push-to-talk  adds  quite  a  bit  of  load.  The  push-to-talk  data  by  condition  over  missions  is 
unreliable  with  performance  all  over.  Flowever,  the  no  push-to-talk  data  show  more  regularity. 
They  demonstrate  that  staying  in  the  same  context  is  better  than  changing  context. 

The  data  also  included  transcripts  of  conversations  among  team  members.  Subjective 
analysis  indicates  that  including  a  communication  component  both  in  the  beginning  of  the 
training  and  before  the  first  mission  was  effective.  However,  additional  (possibly  automatic) 
analysis  of  the  transcripts  of  conversation  may  be  useful  (and  is  being  performed).  In  addition, 
the  results  have  been  examined  for  cross-cultural  individual  difference  factors  and  team  trust 
issues. 


We  are  currently  replicating  the  investigation  again  in  the  SASO  lab  using  a  change  in 
context  in  the  actual  SASO  software  as  described  below. 


Mission  Mission  Mission  Mission  Mission  Mission 

1  2  3  4  5  6 

Missions 


-No  push-to-talk 
-Push-to-talk 


Figure  2.  Experiment  2  results  for  push-to-talk  versus  no  push-to-talk. 
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Figure  3.  Experiment  2  results  for  push-to-talk  results  by  condition. 

Note.  CC=  co-located;  CD  =  co-located  to  distributed;  DC  =  distributed  to  co-located;  DD=  distributed. 


Figure  4.  Experiment  2  results  for  no  push-to-talk  results  by  condition. 

Note.  CC=  co-located;  CD  =  co-located  to  distributed;  DC  =  distributed  to  co-located;  DD=  distributed. 
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Contributions  to  Basic  Research 


In  recent  years,  the  advancement  of  military  technology  has  resulted  in  highly  complex 
skills  that  need  to  be  acquired  and  maintained  to  use  that  technology  (Barry  &  Runyan,  1995). 
Consequently,  the  demand  for  ongoing  innovations  including  distance  training  has  increased. 
Distance  training  has  been  shown  to  be  extremely  cost  effective  and  has  produced  similar 
learning  outcomes.  Although  extensive  research  on  distance  learning  exists,  few  studies  have 
focused  on  group  (collaborative)  distance  training  delivered  by  internet/ Web-based  technologies. 

A  parallel  development  has  occurred  in  the  area  of  team  training.  Even  though  group 
(collaborative)  learning  has  been  the  focus  of  many  studies  (Slavin,  1996)  and  progress  has  been 
made  on  knowledge  measurement  at  the  individual  level,  the  measurement  of  team  knowledge, 
and  team  cognition  in  general,  is  still  in  its  infancy  (Cooke,  Salas,  Cannon-Bowers  &  Stout, 
2000).  Thus,  research  on  how  to  improve  team  distance  learning  can  contribute  significantly  to 
the  field  in  several  areas  including  distance  education  and  across-context  team  transfer. 

Interest  in  distance  education  has  increased  due  to  a  concerted  effort  on  the  part  of  public 
educational  institutions  to  reach  new  populations  of  students  who  might  not  otherwise  be  able  to 
attend.  With  advancements  in  technology,  this  has  been  possible  (particularly  in  the  use  of  Web- 
based  training).  On  the  other  hand,  few  controlled  studies  exist  to  determine  what  is  truly 
beneficial. 

Controlled  studies  for  team  distance  training  could  also  contribute  to  the  transfer 
literature  generally.  Research  has  been  performed  on  individual  transfer  (Singley  &  Anderson). 
Although  specifications  for  transfer  are  delineated,  research  on  context  effects  has  not  been  as 
conclusive  (see  McDaniel,  Anderson,  Einstein  &  O'Halloran,  1989  or  Wickens,  1987).  The 
current  research  project  is  focused  upon  describing  the  effects  of  a  particular  physical  context 
change  on  team  performance  (with  a  focus  on  cognitive  skills,  but  also  some  measurement  of 
social  skills). 

Potential  Army/Military  Applications 

The  applications  to  the  military  were  expected  to  be  in  principles  for  distance  training  and 
transfer  (changing  from  one  situation  to  another)  situations.  These  studies  were  designed  to 
determine  what  conditions  promote  distance  training  and  transfer  and  what  training  is  needed  to 
improve  team  and  distance  transfer.  In  addition,  within  the  studies,  various  assessment  measures 
were  used  that  will  focus  on  individual  and  team  cognitive  and  social  (social-cultural)  skill 
development. 

Final  Summary 

These  experiments  provided  many  hours  of  training  compared  to  studies  usually 
performed  in  university  laboratories.  Here  are  our  findings  and  implications. 

Experiment  1  indicated  the  importance  of  communication  training.  Teams  that  may  be 
composed  of  people  who  do  not  know  each  other  (from  different  parts  of  the  country)  may  need 
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to  become  comfortable  in  communicating  with  each  other  before  a  team  task  can  be  performed. 
Unfortunately,  people  are  often  expected  to  learn  how  to  communicate  through  leam-by-doing  or 
by  exposure. 

In  Experiment  1 ,  the  hardware  was  imposing  and  took  time  to  train.  In  addition,  only 
push-to-talk  was  used.  In  Experiment  2,  we  directly  examined  push-to-talk  versus  no  push-to- 
talk.  The  hardware  was  less  imposing  (more  familiar  and  easier  to  learn).  The  results,  after 
separating  push-to-talk  from  no  push-to-talk,  resembled  what  one  would  expect  -  a  change  of 
physical  context  and  software  context  would  result  in  more  difficulties  than  staying  in  the  same 
context.  Although  push-to-talk  is  a  normal  function  of  technology  in  the  military,  it  is  not  usual 
outside  of  the  military.  Further,  when  people  are  face-to-face,  having  to  press  a  button  to  speak  is 
somewhat  unnatural.  A  more  general  conclusion  is  that  individuals  (and  teams)  need  training  on 
just  the  hardware  aspects  before  trying  to  train  on  the  target  knowledge  available  in  the 
simulations.  One  needs  to  examine  the  aspects  of  the  hardware  (and  software)  to  locate  potential 
difficulties  before  focusing  on  whether  the  goals  of  learning  can  be  achieved. 

The  results  in  both  experiments  were  different;  however,  the  hardware  and  software  were 
also  different.  Note  that  Experiment  1  had  far  fewer  teams.  Even  though  numerous  other 
differences  between  these  experiments  may  have  impacted  the  results,  these  studies  together 
indicate  how  many  different  types  of  context  might  exist  for  the  individual  or  team.  There's  the 
physical  context  (same  room/different  room/different  floors),  the  hardware  context  (large 
imposing/unfamiliar  or  familiar)  and  the  software  context  (SASO's  country  map  &  stories,  UAV 
maps  &  buildings).  If  the  context  (simulation  or  hardware)  is  particularly  compelling,  then  the 
physical  context  (room,  building,  chairs,  etc.)  may  not  play  as  large  a  role.  However,  identifying 
what  the  critical  context  is  for  the  team  or  individual  may  be  the  harder  task. 

Team  performance  can  resemble  individual  performance.  For  interference,  a  decrement 
would  be  expected  at  transfer  to  a  different  context,  but  after  a  little  more  training,  the 
performance  will  jump  back  to  the  original  performance  before  transfer.  For  most  teams,  this 
result  was  found.  Thus,  in  most  cases,  a  team  that  performed  peacekeeping  in  one  country  could 
be  brought  together  in  another  country  with  a  temporary  loss  of  performance.  We  do  not  know 
how  much  of  a  decrement  would  occur  or  how  long  it  would  take  to  bounce  back,  but  at  least  we 
do  know  that  the  training  is  not  lost  permanently. 


24 


The  Integration  of  Implicit  and  Explicit  Knowledge  in  Skill  Learning 


Contract  #:  W74V8H-04-K-0002 
Institution:  Rensselaer  Polytechnic  Institute 


Contract  Dates:  09/2005  to  1 0/2008 
PI:  Ron  Sun 
Co-PI:  Robert  Mathews 
Liaison:  Dan  Horn,  BRU 


Problem(s)/Research  Question(s)  -  How  is  implicit  knowledge  acquired  from  experience 
integrated  with  explicit  knowledge  from  analytical  thinking,  and  what  role  does  this  integration 
play  in  skill  learning  and  in  leading  to  skilled  performance? 

Technical  Barrier(s)  -  The  interaction  of  these  two  types  of  knowledge  is  still  poorly 
understood  and  more  importantly,  the  integration  of  these  two  types  of  knowledge  is  heretofore 
unexplored.  The  process  of  interaction  and  integration  appears  to  be  highly  complex,  varied,  and 
multi-faceted. 

Significance/Impact  for  Basic  Research  -  The  major  contribution  of  this  work  lies  in  coming 
up  with  a  significant  new  theory  that  explains  a  range  of  human  data  (skill  learning  phenomena) 
on  the  basis  of  the  integration  of  implicit  and  explicit  knowledge.  The  integration  of  these  two 
types  of  knowledge  is  heretofore  unexplored  in  the  literature.  Therefore  this  project  will  lead  to 
advancing  the  psychology  of  skill  learning  and  better  understanding  of  this  essential  aspect  of 
human  cognition. 

Potential  transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Future  Force  Warrior  Training  program  (WP  2 1 5)  -  Training  programs  that  facilitate 
rapid  accurate  decision-making;  and 

•  Various  other  training  programs,  including  VICTOR:  Virtual  Individual  and  Collective 
Training  for  Future  Warriors  (WP  233);  SIMAIR:  Simulation-Focused  Collective 
Aircrew  Training  (WP  231);  and  Training  for  Interactive  Distributed  Environments  (WP 
214)  -  How  implicit  and  explicit  training  interact  and  affect  skill  learning  should  be 
relevant  to  all  of  these  programs. 


Overview 


It  has  been  shown  that  high  levels  of  skills  depend  on  both  conceptual  (explicit)  and 
subconceptual  (implicit)  knowledge.  In  our  prior  work,  we  have  demonstrated  that  there  is  often 
a  lack  of  awareness  of  experiential  knowledge  and,  in  fact,  implicit  experiential  knowledge  may 
sometimes  be  incompatible  with  explicit  conceptual  knowledge.  These  problems  create  serious 
challenges  for  training  and  learning  in  the  military  as  well  as  the  civilian  contexts.  The  purpose 
of  this  research  program  is  to  better  understand  the  integration  of  these  two  types  of  knowledge 
in  the  course  of  skill  learning  and  how  their  integration  may  lead  to  better  skill  learning  and 
better  skilled  performance.  Specifically,  we  will  ( 1 )  investigate  the  integration  of  implicit 
knowledge  (from  experiential  learning)  and  explicitly  formulated  conceptual  knowledge  in  the 
development  of  skills  and  expertise,  (2)  explore  when  and  how  conscious  reflection  and  resulting 
conceptual  knowledge  may  be  integrated  with  implicit  knowledge  to  enhance,  rather  than  to 
interfere  with,  experiential  learning,  (3)  develop  computational  models  and  theories  of  human 
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skill  learning  that  capture  the  fine-grained  details  of  the  integration  of  these  two  types  of 
learning,  which  may  be  of  use  in  understanding  and  improving  training,  on  the  basis  of  our 
experimental  results. 

Research  Approach 

This  project  investigates  the  integration  of  implicit  and  explicit  knowledge  in  skill 
learning  through  experimental,  theoretical,  and  computational  means.  In  part,  this  work  develops 
further  a  computational  cognitive  architecture  CLARION,  especially  in  terms  of  capturing  of 
process  details  of  bottom-up  learning  and  top-down  learning.  These  two  processes  represent  two 
hypothesized  ways  of  integrating  implicit  and  explicit  knowledge.  Equally  important  in  this 
project  are  a  set  of  human  experiments  to  explore  different  methods  of  facilitating  the  integration 
in  order  to  enhance  skill  acquisition. 

The  human  experiments  will  use  a  more  complex  process  control  task  than  those 
typically  used  in  the  implicit  learning  literature  that  we  refer  to  as  a  diagnostic  task.  The  task 
involves  several  interrelated  input  and  output  variables.  The  task  captures  real  world  situations  in 
which  an  operator  (e.g.,  a  manager)  is  attempting  to  control  a  main  target  variable  (e.g.,  field 
performance)  but  must  also  monitor  several  other  variables  to  be  sure  they  remain  in  acceptable 
levels  (e.g.,  desire  to  re-enlist,  or  morale).  It  also  corresponds  to  situations  where  sequences  of 
decisions  must  be  made  (selecting  one  or  more  input  variables  to  use  on  each  case)  before 
feedback  begins  arriving  about  earlier  decisions.  This  is  analogous  to  a  manager  trying  different 
treatments  to  improve  workers'  performance  and  waiting  to  see  results,  then  making  the  decision 
to  continue  current  treatment  or  to  try  something  new.  It  also  permits  study  of  the  effects  on 
learning  of  individual  variability  in  treatment  outcomes  (e.g.,  not  all  workers  respond  alike  to  the 
same  treatment).  While  the  task  is  dynamic,  it  provides  feedback  by  discrete  time  steps  (i.e.,  after 
each  round  of  decisions,  feedback  is  provided  on  the  effect  of  the  treatment  decisions  on  each 
individual  case).  This  division  of  performance  into  discrete  time  steps  provides  the  opportunity 
to  reveal  learning  strategies  and  to  view  changes  in  the  learning  processes  over  time. 

In  addition,  the  task  contains  a  special  model-building  screen  to  develop  and/or  to  reveal 
participants*  explicit  hypotheses  about  the  relationships  among  the  variables.  Participants  are 
told  that  their  goal  is  to  figure  out  the  relationships  among  input  and  output  variables  and 
correctly  map  these  relationships  on  the  model-building  screen.  Responses  on  this  screen  reveal 
participants"  conscious  (explicit)  knowledge  about  strength  and  direction  of  relations  among 
input  and  output  variables  and  indicate  their  confidence  in  their  responses.  Participants  also  will 
indicate  what  they  plan  to  investigate  in  the  next  sequence  of  task  performance. 

In  various  experiments,  opportunities  to  switch  from  the  diagnostic  task  screen  (assigning 
treatments  to  individuals  and  obtaining  feedback  from  the  past  decisions  on  all  output  variables) 
to  the  model  building  screen  will  be  controlled.  In  some  experiments,  participants  will  perform 
the  task  at  a  fast  pace  (they  must  respond  every  six  seconds)  to  build  implicit  knowledge  of  the 
task  while  limiting  time  available  to  develop  conscious  hypotheses  to  test.  Then,  in  a  second 
phase  of  the  experiment,  they  will  be  allowed  access  to  both  screens  to  study  how  they  translate 
their  implicit  knowledge  of  the  task  into  an  explicit  model  of  the  task.  In  other  experiments, 
participants  will  be  encouraged  to  engage  explicit  learning  processes  to  leam  how  to  perform  the 
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task.  They  will  be  required  to  formulate  and  test  their  hypotheses  by  switching  back  and  forth 
from  the  model  screen  to  the  task  screen.  These  experiments  will  provide  us  with  snapshots  of 
how  people  attempt  to  implicitly  and  explicitly  leam  the  task.  Subsequent  experiments  will  test 
various  types  of  integrated  training  such  as  providing  initial  (partially  reliable)  explicit 
knowledge  about  the  relations  among  variables.  Other  variables  to  be  studied  will  include  the 
presence  or  absence  of  hints  or  cognitive  aids  (e.g.,  a  history  screen  that  allows  learners  access  to 
previous  inputs  and  their  effects  on  a  given  individual),  and  type  of  task  goal  -  whether  the  task 
goal  is  performance  based  (e.g.,  getting  all  scores  into  the  target  zones)  versus  discovery  oriented 
(e.g.,  testing  the  effects  of  all  treatment  combinations). 

Accomplishments 

This  contract  began  in  September  of  2005,  starting  with  a  major  development  of  software 
for  experiments.  We  are  now  completing  the  first  series  of  experiments  using  the  new  software. 

We  have  already  made  an  interesting  discovery.  When  a  treatment  has  a  strong  positive 
effect  on  the  main  target  output  variable  but  also  has  a  negative  “side  effect”  on  one  of  the  other 
output  variables,  participants  are  generally  unaware  of  the  side  effect  but  discount  (reduce)  the 
estimated  positive  effect  of  the  treatment  on  the  main  target  output  variable.  This  discounting 
effect  is  in  proportion  to  the  size  of  the  negative  side  effect  and  does  not  go  away  even  when 
participants  are  instructed  to  pay  special  attention  to  the  output  variable  that  has  the  side  effect. 
This  effect  appears  to  be  unconscious  (implicit)  but  influences  conscious  estimates  of  effect  size 
on  the  main  target  output  variable.  We  are  now  conducting  additional  experiments  to  see  if  this 
effect  increases  or  diminishes  when  training  emphasizes  explicit  (responding  to  the  model 
screen)  or  implicit  (just  trying  to  control  the  variables)  learning. 

During  this  time,  we  also  wrote  and  published  an  article  involving  two  artificial  grammar 
experiments  that  demonstrates  the  importance  of  including  both  bottom-up  and  top-down 
learning  mechanisms  to  adequately  account  for  artificial  grammar  learning.  Across  two 
experiments,  we  trained  participants  using  a  string  edit  task  and  highlighted  relatively  low-level 
(letters),  medium-level  (chunks),  or  high-level  (structural,  i.e.,  grammar  diagram)  information  to 
increase  the  efficiency  of  grammar  acquisition.  In  both  experiments,  participants  who  had 
structural  information  available  during  training  generated  more  highly  accurate  strings  during  a 
cued  generation  test  than  other  conditions,  with  equivalent  speed.  Experiment  2  revealed  that 
structural  information  only  enhanced  acquisition  when  relevant  features  were  highlighted  during 
the  task  using  animation.  We  suggest  that  two  critical  components  for  producing  enhanced 
performance  from  provided  model-based  (explicit)  knowledge  involve:  (1)  using  the  model 
(explicit  knowledge)  to  acquire  experience-based  (more  implicit)  knowledge  rather  than  using  a 
representation  of  the  model  to  generate  responses  and  (2)  receiving  that  explicit  knowledge 
precisely  when  it  is  needed  during  training.  Our  results  were  most  supportive  of  the  notion  that  a 
combination  of  top-down  (model-based,  explicitly  driven)  and  bottom-up  (experience-based, 
implicitly  driven)  processing  might  be  a  particularly  powerful  way  to  increase  the  acquisition  of 
expertise  (e.g.,  Sun,  2002;  Sun  et  al.,  2005).  However,  our  work  also  suggests  integrating  these 
types  of  knowledge  is  not  easy.  It  is  clear  that  simply  providing  information  about  the  underlying 
structure  of  the  grammar  and  exposing  participants  to  exemplars  during  training  is  not  sufficient. 
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Simulations  of  various  combinations  of  explicit  and  implicit  learning  have  also  been 
conducted,  and  we  are  in  the  process  of  analyzing  the  results  and  developing  new  theoretical 
frameworks. 

Contributions  to  Basic  Research 

The  major  contribution  of  this  work  lies  in  coming  up  with  a  significant  new  theory  that 
explains  a  range  of  human  data  (skill  learning  phenomena)  on  the  basis  of  the  integration  of 
implicit  and  explicit  knowledge.  One  product  of  this  effort  is  a  conceptual  framework,  which 
addresses  the  ways  with  which  these  two  types  of  knowledge  may  be  integrated  to  produce 
expertise.  This  theory  (the  CLARION  model)  suggests  that  performance  can  be  controlled  by 
either  a  subconceptual  knowledge  base  (implicit  mode)  or  a  symbolic  conceptual  mental  model 
(explicit  mode).  Implicit  control  is  fast  but  prone  to  error,  particularly  in  early  stages  of  skill 
acquisition.  Explicit  control  is  more  accurate  but  slow  to  apply,  and  prone  to  loss  by  forgetting 
over  a  retention  interval.  Their  integration  may  lead  to  better  learning  and  better  performance. 

A  computational  cognitive  architecture,  CLARION,  which  is  substantially  different  from 
other  existing  cognitive  architectures,  will  be  further  developed  in  this  work  to  simulate  and 
capture  a  wide  range  of  quantitative  data  that  are  related  to  the  integration  of  the  two  types  of 
knowledge,  based  on  the  above  ideas.  This  will  help  us  to  explain  (and  eventually  to  predict) 
training  and  learning  processes.  We  will  carry  out  simulations  to  generate  new  insights  and 
interpretations  that  can  further  explicate  the  integration  of  implicit  and  explicit  processes.  These 
outcomes  (data,  models,  and  theories)  will  likely  provide  a  new  perspective  on  skill  learning. 

Our  models  and  theories  will  be  useful  in  better  understanding  human  skill  learning  theoretically 
as  well  as  in  helping  to  improve  training  processes  in  practice. 

Potential  Army/Military  Applications 

This  research  links  to  applied  programs  on  training  and  learning.  Specifically,  our  basic 
research  centers  on  understanding  learning  and  cognition  in  terms  of  the  integration  of  implicit 
and  explicit  processes,  and  thus  it  involves  testing  and  developing  learning  and  training 
principles  that  may  be  used  to  enhance  human  learning.  The  goals  of  the  basic  research  align 
with  those  of  the  applied  programs.  Both  focus  on  training  effectiveness.  Hence,  work  from  our 
basic  research  could  inform  and  be  applicable  to  work  in  the  applied  programs.  For  example,  our 
work  may  offer  training  principles  that  enhance  training  effectiveness,  which  can  then  be 
incorporated  into  an  automated  tutoring  system.  There  are  of  course  many  other  ways  in  which 
the  results  of  our  basic  research  program  could  be  employed  in  the  applied  programs'  systems, 
tools,  and  other  products. 

We  expect  that  Army  applications  of  this  research  will  be  in  the  area  of  training  and 
human  performance.  A  basic  problem  for  the  Army  is  how  to  ensure  that  novices  in  a  Military 
Occupational  Specialty  (MOS)  move  quickly  to  more  advanced  performance  (and  perhaps  to 
expertise)  as  a  result  of  their  training.  In  addition,  most  training  focuses  on  teaching  conceptual 
(explicit)  knowledge  rather  than  setting  up  the  opportunity  for  substantial  experiential  (mostly 
implicit)  knowledge.  While  this  may  be  appropriate  for  some  specialties,  some  other  specialties 
involve  working  with  complex  systems  that  are  better  learned  initially  through  extensive 
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experiences  than  with  lectures  or  textbook  lessons.  As  service  personnel  gain  in  expertise,  it  may 
often  then  become  appropriate  to  focus  more  on  conceptual  (explicit)  knowledge.  In  this  process, 
the  integration  of  the  two  kinds  of  knowledge  is  expected  to  be  highly  important.  Our  research 
will  lead  to  some  initial  guidelines  about  how  to  best  facilitate  knowledge  integration  and  thus 
learning  and  performance. 

Future  Plans 

Our  contract  began  towards  the  end  of  Fiscal  Year  2005.  We  have  acquired  new 
equipment  and  developed  new  software  for  conducting  human  experiments.  Now  on  that  basis, 
we  have  begun  to  conduct  the  first  set  of  experiments  as  specified  in  our  proposal. 

The  new  human  experiments  use  a  more  complex  process  control  task  than  those 
typically  used  in  the  implicit  learning  literature.  It  has  five  input  variables  that  can  interact  on 
five  output  variables.  This  task  (the  diagnostic  task)  has  a  direct  analog  to  real  world  situations 
such  as  making  managerial  decisions  or  clinical  practice  in  medicine.  In  these  situations,  learners 
must  contend  with  multiple  sources  of  variability  (i.e.,  unreliable  explicitly  provided 
information,  individual  variability  in  treatments,  and  client  turnover).  This  task  will  allow  us  to 
study  effects  of  these  natural  sources  of  variability  on  implicit  and  explicit  learning  outcomes, 
and  their  integration.  Our  previous  studies  have  found  implicit  learning  to  be  more  tolerant  of 
learning  under  noisy  conditions.  We  also  plan  to  study  the  types  of  errors  learners  make  under 
these  conditions  and  how  to  prevent  them. 

In  addition,  the  diagnostic  task  contains  a  special  model-building  screen  to  develop  and 
reveal  explicit  hypotheses  about  the  relationships  among  the  variables  over  time.  In  various 
experiments,  opportunities  to  switch  from  the  task  screen  to  the  model  building  screen  will  be 
controlled.  In  some  experiments,  participants  will  perform  the  task  at  a  fast  pace  (must  respond 
every  six  seconds)  to  build  implicit  (without  explicit)  knowledge  of  the  task.  Then,  in  a  second 
phase  of  the  experiment,  they  will  be  allowed  access  to  both  screens  to  study  how  they  translate 
their  implicit  knowledge  of  the  task  into  an  explicit  model  of  the  task.  In  other  experiments, 
participants  will  be  encouraged  to  engage  explicit  learning  processes  to  leam  how  to  perform  the 
task.  They  will  be  required  to  formulate  and  test  their  hypotheses  by  switching  back  and  forth 
from  the  model  screen  to  the  task  screen.  These  experiments  will  provide  us  with  snapshots  of 
how  people  attempt  to  implicitly  and  explicitly  leam  the  task.  Other  experiments  will  test  various 
ways  of  integrating  implicit  and  explicit  training.  Other  variables  studied  will  include  various 
hints  to  help  build  accurate  models  and  whether  task  feedback  is  performance  (error  detection) 
focused  versus  discovery  (creativity)  focused. 

Simultaneously,  we  will  develop  the  basic  outlines  of  computational  models  for  capturing 
various  methods  for  integrating  implicit  and  explicit  knowledge  in  skill  learning,  namely  various 
bottom-up  and  top-down  learning  algorithms.  We  will  then  test  them  against  human  data  to  be 
generated  from  human  experiments.  We  also  will  work  on  further  cognitive  architecture 
developments. 

We  will  continue  to  prepare  manuscripts  of  theoretical  and/or  experimental  papers, 
arguing  for  and  developing  our  unique  approach. 
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BRU  RESEARCH  OBJECTIVE  #2:  PROVIDE 
FUNDAMENTAL  KNOWLEDGE  TO  IMPROVE  LEADER 
AND  TEAM  PERFORMANCE 


Research  under  this  objective  provides  concepts  and  methods  for 
accelerating  leader  development,  understanding  and  developing  leader  adaptability 
and  flexibility,  and  discovering  and  testing  the  basic  cognitive  principles  that 
underlie  effective  leader-team  performance.  Understanding  the  dynamics  of  small 
group  leadership  in  face-to-face  and  distributed  team  environments  is  critical  to 
this  research  objective  as  well. 
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Problem(s)/Research  Question(s)  -  What  predicts  individuals '  adaptive  performance  in 
dynamic  settings?  What  predicts  teams '  adaptive  performance  in  dynamic  settings?  What  can 
team  leaders  do  to  enhance  adaptive  individual  and  team  performance? 

Technical  Barrier(s)  -  While  research  on  leadership  is  abundant,  as  is  research  on  teams, 
surprisingly  few  researchers  have  focused  specifically  on  team  leadership.  Further,  it  is 
challenging  to  identify,  gain  access  to,  investigate,  and/or  stimulate  truly  dynamic  work  settings. 

Significance/Impact  for  Basic  Research  -  Our  findings  will  help  to  clarify  the  nature  and 
antecedents  of  adaptive  individual  and  team  performance  in  dynamic  settings,  expanding 
research  on  (1)  leadership;  (2)  team  effectiveness;  and  (3)  dynamic  work  contexts. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  1 03); 

•  Future  Force  Warrior  Training  (WP  2 1 5); 

•  Assessing  and  Developing  Leaders  Within  Special  Operations  Units  (WP  218); 

•  Accelerating  Leader  Development  (WP  269); 

•  Accelerating  Leader  Development  (WP  285); 

•  SELECT21 :  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force 
Soldier  (WP  257)  and  ATO:  IV.HS.2002.01;  and 

•  FUTURE  TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  211). 


Overview 


Our  research  was  designed  to  illuminate  the  influence  of  individual  differences,  team 
characteristics,  and  leadership  on  individual  and  team  performance  in  dynamic  work 
environments.  Dynamic  settings  are  work  environments  in  which: 

1 .  Diverse  specialists  work  in  teams  to  perform  highly  interdependent  tasks; 

2.  Tasks  are  high  in  novelty,  uncertainty,  and  urgency; 

3.  The  pace  of  work  is  varied  and  unpredictable  as  emergency  events  occur  frequently  but  at 
unpredictable  times; 

4.  Team  composition  (i.e.,  team  membership)  changes  frequently,  such  that  team  members 
may  not  know  many  of  the  individuals  on  their  team;  and 

5.  Effective  task  performance  is  dependent  on  team  member  learning,  development,  and 


adaptation. 
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Theory  and  research  regarding  the  factors  that  allow  individuals  and  teams  to  thrive  in 
such  settings  are  extremely  limited.  Our  research  helps  to  fill  this  gap  in  the  literature. 
Ultimately,  our  work  will  inform  training  specifications  for  leader  development  so  that  Army 
leaders  are  equipped  with  the  skills  needed  to  shape  and  influence  team  learning,  development, 
and  adaptability. 

Research  Approach 

We  conducted  five  interrelated  research  projects,  combining  research  methods  and 
approaches — including  comprehensive  foundational  literature  reviews,  theory  development, 
experimental  research  in  a  laboratory  setting,  qualitative  case  study  research,  and  longitudinal 
survey  research  in  the  field — to  build  new  understanding  of  the  ways  in  which  leaders  may 
enhance  team  learning,  coordination,  and  adaptive  performance  in  dynamic  work  environments. 

More  specifically,  we  collected  qualitative  data  in  a  real-life,  dynamic  work  setting:  the 
Maryland  Shock  Trauma  Center  (STC).  STC  is  the  hub  of  Maryland's  system  of  emergency 
trauma  care,  treating  over  6,000  victims  of  traumatic  injury  each  year.  We  complemented  the 
real-world  insights  that  we  gained  in  the  STC  with  rigorous,  experimental  research  in  the  MSU 
ADAPT  Lab  (Accelerated  Development  and  Adaptive  Performance  Training).  Our  experimental 
work  was  based  on  the  theoretical  foundation  of  self-regulation  as  a  means  to  examine  the 
psychological  mechanisms  underlying  learning,  motivation,  and  performance.  We  extended  and 
enriched  these  efforts  by  conducting  detailed  reviews  of  the  literature,  engaging  in  theory¬ 
building,  and  studying  the  interactive  effects  of  team  composition  and  leader  behavior  in  a 
longitudinal  survey  study  of  national  service  teams. 

Accomplishments 

In  our  experimental  research  in  the  MSU  ADAPT  Lab  on  complex  skill  acquisition,  we 
found  that  orienting  team  members  toward  learning  (via  goal  content  and  goal  frames)  is  a  potent 
means  to  enhance  the  nature,  focus,  and  quality  of  self-regulatory  processes,  learning,  and 
performance  outcomes.  Principles  from  this  research  may  guide  the  development  of  simulation 
tools  and  guidelines  for  team  leaders.  For  more  detailed  information  about  this  research,  see: 

Kozlowski,  S.  W.  J„  &  Bell,  B.  S.  (2006).  Disentangling  achievement  orientation  and 

goal  setting:  Effects  on  self-regulatory  processes.  Journal  of  Applied  Psychology,  91. 

900-916. 

Our  foundational  review  on  team  effectiveness  documented  several  critical  cognitive, 
motivational,  and  affective  team  processes  that  contribute  to  team  effectiveness  and  pointed  to 
leadership  and  team  development  as  leverage  points  for  enhancing  team  effectiveness.  To  read 
this  review  of  the  team  effectiveness  literature,  see: 

Kozlowski,  S.  W.  J.,  &  Ilgen,  D.  R.  (2006).  Enhancing  the  effectiveness  of  work  groups 

and  teams  (Monograph).  Psychological  Science  in  the  Public  Interest ,  7,  77-124. 

Our  conceptual  analysis  of  dynamic  team  leadership  addressed  gaps  in  the  mainstream 
leadership  literature.  Our  theory  of  dynamic  team  leadership  specifies  how  team  leaders  can 
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shape,  influence,  and  harness  self-regulatory  processes  to  guide  team  development  and  the 
emergence  of  key  team  processes  needed  for  adaptation  and  effectiveness.  The  theory  is 
presented  in: 

Kozlowski,  S.  W.  J.,  Watola,  D„  Nowakowski,  J.  M„  Kim,  B„  &  Botero,  I.  (in  press). 

Developing  adaptive  teams:  A  theory  of  dynamic  team  leadership.  In  E.  Salas,  G.  F. 

Goodwin,  &  C.  S.  Burke  (Eds.),  Team  effectiveness  in  complex  organizations:  Cross- 

disciplinary  perspectives  and  approaches  (SIOP  Frontiers  Series).  Mahwah,  NJ:  LEA. 

Our  qualitative  research  conducted  in  an  emergency  trauma  treatment  center  underscored 
the  importance  of  dynamic  team  leadership.  Senior  team  leaders  in  this  setting  engaged  in 
"dynamic  delegation,”  repeatedly  delegating  the  active  leadership  role  to,  and  withdrawing  the 
active  leadership  role  from,  more  junior  leaders  of  the  team  in  response  to  rapidly  changing  task 
contingencies.  Dynamic  delegation  allowed  teams  to  perform  reliably  while  also  building  their 
novice  team  members’  skills.  This  study  appears  in: 

Klein,  K.  J.,  Ziegert,  J.  C.,  Knight,  A.  P.,  &  Xiao,  Y.  (2006).  Dynamic  delegation: 

Shared,  hierarchical,  and  deindividualized  leadership  in  extreme  action  teams. 

Administrative  Science  Quarterly.  50,  590-621. 

Finally,  our  longitudinal  investigation  of  leadership,  diversity,  and  team  conflict, 
highlighted  the  need  for  leader  responsiveness  to  their  teams'  deep  diversity.  When  team 
members'  values  differ,  leaders  who  structure  their  teams’  tasks  (by  assigning  tasks,  setting 
deadlines,  specifying  standards,  etc.)  may  be  particularly  effective  in  preventing  team  conflict 
and  facilitating  rapid  coordination  and  adaptation.  For  more  information,  see: 


Klein,  K.  J.,  Knight,  A.  P„  Ziegert,  J.  C.,  Lim,  B.  C.,  &  Saltz,  J.  L.  (2007.)  When  team 
members  ’  values  differ:  The  moderating  effects  of  team  leadership.  Working  paper. 
University  of  Pennsylvania,  The  Wharton  School. 

Contributions  to  Basic  Science 

Our  research  sheds  new  light  on  the  psychological  mechanisms  underlying  individual  and 
team  self-regulation  in  dynamic  work  settings.  Further,  our  research  identifies  leader  behaviors 
and  functions  (e.g.,  goal  clarification,  feedback,  coaching)  that  enhance  individual  and  team  self¬ 
regulation  and  thereby  enhance  individual  and  team  learning  and  development  in  dynamic 
settings.  Current  research  on  leadership  and  on  individual  and  team  performance  is  dominated  by 
relatively  static,  structural  approaches.  Our  research  highlights  the  dynamic  nature  of  individual, 
team,  and  leader  performance  required  in  dynamic  work  settings. 

Potential  Army/Military  Applications 

Our  research  findings  have  implications  for  military  leadership,  training,  selection,  and 
team  (e.g.,  platoon)  design.  The  knowledge  we  are  generating  can  be  applied  to  improve 
leadership  (in  action),  leader  development  (training  specifications  for  leader  capabilities),  and 
training  design  and  delivery  (what  we  develop  can  be  delivered  by  leaders  in  action  or  by 
technology-based  training  systems).  Our  findings  can  also  inform  the  selection  of  Soldiers  for 
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work  in  dynamic  team  settings,  shedding  light  on  individual  differences  that  predict  effective 
learning  and  performance  in  dynamic  settings.  Finally,  analyses  of  the  role  that  leaders  may  play 
in  attenuating  the  potentially  negative  effects  of  team  values  diversity  clarifies  the  leader 
behaviors  likely  to  be  most  effective  when  Soldiers'  values  differ. 

Final  Summary 

Adaptive  team  performance  is  multi-determined.  Adaptive  individual  and  team 
performance  reflects  the  characteristics  of  individual  team  members ,  most  notably  their  learning 
goal  orientation.  Further,  adaptive  individual  and  team  performance  reflects  the  behaviors  of 
team  leaders.  Team  leaders  may  cultivate  team  members'  learning  orientation  and  skill 
acquisition  by  coaching  and  training  their  team  members,  by  providing  structure,  and  by  enacting 
dynamic  delegation.  In  highly  effective  teams,  team  processes — including  team  efficacy, 
transactive  memory,  and  team  cohesion — facilitate  adaptive  performance.  Finally,  skilled  team 
leaders  enhance  their  teams'  adaptive  performance  by  adjusting  and  matching  their  leadership 
behaviors  to  critical  contingencies ,  including  changes  in  team  composition  and  shared  expertise, 
values  diversity,  and  shifts  in  the  team's  task  engagement  cycle. 
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Self-Initiated  Development  of  Leadership  Capabilities: 

Toward  Establishing  the  Validity  of  Key  Motivational  Constructs 

and  Assessment  Tools 

Contract  #:  W74V8H-05-K-0001  Contract  Dates:  1 1/01/2004  to  10/31/2007 

Institution:  Georgia  State  University  PI:  Todd  J.  Maurer 

Liaison:  Larry  Laffitte,  LDRU 

Problem (s)/Research  Question(s)  -  The  problem  addressed  in  this  research  is  to  identify  and 
validate  new  individual  difference  characteristics  that  are  relevant  to  motivating  leadership 
development,  as  well  as  ways  of  measuring  those  differences  in  people. 

Technical  Barrier(s)  -  Little  is  known  about  predicting  leadership  development  behavior 
because  most  research  has  been  directed  at  predicting  the  performance  of  leaders. 

Significance/Impact  for  Basic  Research  -  New  behavioral  constructs  are  needed  that  will  be 
useful  in  understanding,  predicting  and  ultimately  enhancing  leadership  development.  It  is 
expected  that  hypothesized  relationships  of  these  leadership  development  constructs/measures 
with  each  other  and  with  additional  predictors  will  help  to  establish  their  validity  and  potential 
usefulness  in  this  domain. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  LEADERSHIP:  Developing  leaders  in  a  changing  army  (WP  103); 

•  Accelerating  Leader  Development  (WP  269); 

•  Accelerating  Leader  Development  (WP  285);  and 

•  SELECT2 1 :  Selection,  Classification  and  Performance  Metrics  for  the  Objective  Force 
(WP  257);  ATO:  IV.HS.  2002.01. 

Overview 

The  Army  (and  other  organizations)  must  have  leaders  or  potential  leaders  who 
continuously  pursue  the  development  of  leadership  skills  and  who  are  flexible  and  adaptable 
through  their  involvement  in  self-initiated  development  and  learning  experiences.  The  Army 
cannot  rely  solely  on  mandated  training  or  learning  experiences — it  needs  leaders  or  potential 
leaders  to  be  motivated  to  pursue  self-initiated  development  of  leadership  skills  and 
characteristics.  Total  development  must  be  motivated  and  initiated  from  within  the  person.  But 
little  is  known  about  predicting  leadership  development  behavior  because  most  research  has  been 
directed  at  predicting  the  performance  of  leaders.  Leadership  development  and  leader 
performance  are  not  the  same  thing.  Likewise,  motivation  to  be  a  leader  and  motivation  to 
develop  leadership  skills  are  not  the  same.  The  Army  needs  a  better  understanding  of  leadership 
development  and  those  behavioral  constructs  and  assessment  methods  that  are  relevant  to 
leadership  development,  especially  those  that  involve  individual  sustained  effort  to  develop 
leadership  attributes.  Basic  research  that  addresses  these  issues  can  ultimately  lead  to  applied 
research  which  enables  better  selection  and  development  of  leaders. 
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A  key  first  step  in  this  process  is  to  develop  behavioral  variables/constructs  and 
assessment  methods,  including  a  network  of  related  variables,  that  would  help  establish  the 
validity  and  relevance  of  these  new  variables  in  this  domain.  The  variables  will  be  explored 
within  a  system  of  related  constructs,  including  antecedents  and  consequences,  to  establish  the 
relevance  to  other  variables  within  the  leadership  development  domain. 

There  will  be  two  general  sets  of  variables  examined  in  this  research.  The  first  set  of 
variables  includes  variables  being  introduced  here  as  new  to  the  leadership  domain.  This  set 
includes:  motivation  for  leadership  development,  self-efficacy  (or  self-confidence)  for  leadership 
development,  and  beliefs  about  the  improvability  of  leadership  skills/attributes  (e.g.  beliefs  that 
leadership  can  be  developed  vs.  leadership  is  bom).  In  addition,  participation  in  leadership 
development  activities  for  the  purpose  of  leaming/developing  leadership  skills  also  will  be 
examined.  The  second  set  of  variables  to  be  examined  are  traditional  personality  variables,  as 
well  as  selected  variables  from  the  employee  development  literature,  that  can  be  examined  as 
predictors  of  these  new  variables. 

In  general,  it  is  expected  that  hypothesized  relationships  of  these  leadership  development 
variables  with  each  other  and  with  additional  predictors  will  help  to  establish  their  validity  and 
relevance  in  the  leadership  development  domain.  Constructs  relevant  to  leadership  development 
(e.g.,  self-efficacy  for  leadership  development,  motivation  for  leadership  development)  will  be 
distinguished  theoretically  and  empirically  from  constructs  relevant  to  leadership  performance 
(e.g.,  self-efficacy  for  leadership,  motivation  to  lead). 

It  is  hypothesized  that  (Hypothesis  1)  self-efficacy  for  leadership  development  (SELD) 
will  predict  motivation  to  develop  leadership  skills  (MTDL),  and  that  (Hypothesis  2)  self- 
efficacy  for  leadership  (SEL)  will  predict  motivation  to  lead  (MTL).  The  corresponding  self- 
efficacy  constructs  will  have  higher  predictive  validity  for  the  corresponding  motivational 
constructs  (e.g.,  SELD->  MTDL;  SEL  ->  MTL).  Hypothesis  3  is  that  leadership  improvability 
beliefs  (1BL)  will  predict  SELD,  and  hypothesis  4  is  that  IBL  will  predict  SEL.  although  it  will 
have  a  higher  relationship  with  SELD  due  to  their  closer  conceptual  relationship.  Further, 
(Hypothesis  5,)  MTL  will  predict  MTDL,  but  this  relationship  will  be  greater  for  those 
individuals  who  have  higher  IBL.  In  other  words,  those  who  are  motivated  to  lead  also  will  be 
more  motivated  to  develop  leadership  skills;  however,  this  will  be  more  true  for  those  individuals 
who  have  higher  beliefs  that  leadership  skills/attributes  are  improvable/acquirable.  Therefore, 

IBL  will  moderate  the  relationship  between  MTL  and  MTDL.  Also,  (Hypothesis  6),  MTDL  will 
predict  participation  in  leadership  development  activities  directly.  MTL  will  only  predict 
leadership  development  activities  through  MTDL  and  not  directly;  that  is,  MTDL  acts  as  a 
mediator  between  MTL  and  leadership  development  activities. 

In  terms  of  the  traditional  personality  variables,  it  is  expected  that  openness  to  experience 
(Hypothesis  7)  will  have  a  relationship  with  motivation  to  develop  leadership  (MTDL)  skills  but 
not  with  motivation  to  lead  (MTL).  Prior  research  did  not  find  a  relationship  between  motivation 
to  lead  and  openness.  However,  it  did  find  a  relationship  between  extraversion  and  motivation  to 
lead,  and  so  that  is  expected  here  (Hypothesis  8).  In  addition,  it  is  expected  (Hypothesis  9)  that 
the  traditional  employee  development  constructs  (e.g.,  self-perceptions  that  the  person  possesses 
the  qualities  needed  for  leaming/developing,  perceived  need  for  development  of  one's  own 
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skills,  social  support  for  leaming/developing,  perceived  benefits  of  developing  skills)  will  relate 
to  the  development-oriented  constructs  (e.g.  motivation  to  develop,  self-efficacy  for 
development),  but  not  motivation  to  lead  and  self-efficacy  for  leadership. 

Research  Approach 

A  two-wave  survey  method  will  be  used  to  collect  data  from  participants.  The  source  of 
subjects  will  be  an  on-line  research  pool  through  StudyResponse.com.  As  introduced  at  the  2003 
annual  meeting  of  the  Society  for  Industrial  and  Organizational  Psychology,  StudyResponse  is  a 
service  that  matches  researchers  with  participants  willing  to  receive  solicitations  to  complete 
surveys.  Registered  participants  (currently  N  >  46,000)  have  varied  demographic  characteristics 
including  racial  makeup,  gender,  education,  and  a  varied  list  of  occupations.  Working  adults  of  a 
wide  range  of  ages  will  be  included,  which  will  allow  the  investigation  to  focus  on  both  younger 
and  more  mature  adults. 

Administrators  of  this  service  did  a  benchmarking  study  recently  by  replicating  the  topic 
matter  of  a  national  poll  conducted  with  systematic  sampling  by  a  large  polling  company.  The 
researchers  found  that  the  results  from  the  StudyResponse  panelists’  responses  corresponded 
with  the  results  obtained  in  the  national  poll  of  opinions  within  just  a  few  percentage  points  of 
error  on  just  about  every  question  asked.  This  suggests  that  the  results  provided  by  this  sampling 
approach  are  reasonably  representative  of  data  provided  by  traditional  and  systematic 
sampling/polling  procedures.  This  Internet  panel  approach  provides  a  viable  approach  as  a  sound 
sampling  strategy. 

The  two  sets  of  variables  described  above  will  be  examined  in  the  following  way.  First, 
the  relationships  among  the  leadership  development  constructs  will  be  tested  when  data  for  the 
total  set  of  these  measures  is  collected  in  two  time  periods.  By  collecting  data  for  these  measures 
twice,  this  allows  for  a  retest  interval  to  examine  the  stability  of  the  constructs  across  time  and 
also  for  prospective  prediction  of  some  of  the  constructs  onto  others  at  a  later  time.  For  example, 
self-efficacy  for  leadership  development  at  time  1  can  be  used  to  predict  motivation  to  develop 
leadership  skills  at  both  time  1  and  time  2.  Likewise,  motivation  to  develop  leadership  skills  can 
be  used  to  predict  not  only  participation  in  leadership  development  activities  at  time  1  but  also  at 
time  2.  So  the  relationships  among  the  leadership  constructs  can  be  examined  when  the  data  are 
all  collected  at  time  1 .  They  can  also  be  examined  when  the  data  are  collected  again  at  time  2  and 
the  stability  and  prediction  across  time  can  also  be  tested. 

The  second  set  of  variables  (the  traditional/existing  constructs)  will  also  be  examined  in 
relation  to  the  leadership  variables.  These  data  will  also  be  collected  at  time  1  and  examined  in 
relation  to  the  leadership  constructs  at  time  1 ,  and  also  the  leadership  constructs  that  are 
measured  again  at  time  2.  This  will  also  allow  prospective  prediction  by  some  of  the  traditional 
constructs  onto  others  at  a  later  time.  For  example,  personality  can  be  used  to  predict  self- 
efficacy  for  leadership  development  measured  at  both  time  1  and  at  time  2. 
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Accomplishments 

In  the  first  year  (year  1  of  a  3-year  project),  the  literature  was  reviewed  to  determine  the 
content  of  the  newly  prepared  leadership  development  measures.  Forty-seven  leadership 
attributes  were  identified.  These  attributes  will  be  used  to  collect  ratings  for  leadership 
improvability  beliefs,  self-efficacy  for  leadership  development,  and  motivation  for  leadership 
development.  Corresponding  and  more  generally- worded  measures  of  leadership 
development/improvability  (as  opposed  to  the  development/improvability  of  specific  leadership 
attributes)  were  also  created.  These  measures  and  the  measures  for  the  traditional/existing 
personality  and  employee  development  variables  were  then  reviewed  for  wording,  scaling,  and 
formatting.  Several  iterations  and  revisions  were  done.  A  computerized/Web  version  of  the 
measures  was  developed  and  reviewed,  and  the  system  was  tested  to  ensure  the  accurate  capture 
of  responses  from  the  survey  to  the  database.  A  pilot  investigation  was  then  conducted  to  test  the 
surveys  and  the  data  collection  process.  Over  271  participants  responded  to  all  parts  of  this  pilot 
investigation. 

In  year  two,  the  data  from  the  pilot  were  analyzed  and  the  instruments  examined  to 
determine  if  any  modifications  needed  to  be  made  to  the  measures  and  Web  survey  prior  to  the 
full  data  collection.  In  this  pilot  study,  in  addition  to  testing  the  measures  and  data  collection 
procedure,  we  obtained  some  preliminary  findings  that  nicely  set  up  the  full  study  that  is  yet  to 
come.  In  the  pilot  study,  motivation  to  develop  leadership  was  empirically  distinguished  from 
motivation  to  lead.  Detailed  statistical  and  psychometric  analyses  supported  the  validity  of  the  new 
leadership  development  motivational  measures  so  far.  The  leadership  development  constructs 
predicted  leadership  development  activity  reports  better  than  the  leadership  measures.  Analyses 
further  suggested  that  being  attracted  to  leadership  for  special  rewards  or  extrinsic  reasons  is 
unrelated  to  the  motivation  to  develop  leadership  capability,  while  being  attracted  to  leadership  for 
intrinsic  or  social  responsibility  reasons  is  significantly  related  to  motivation  to  develop  leadership. 
Of  three  types  of  leadership  skills  (task,  people,  and  transformational),  transformational  was  rated 
most  difficult  to  improve  and  participants  were  least  motivated  to  develop  those  skills.  The  results  of 
the  pilot  study  will  be  presented  at  an  international  conference  in  April  of  2007.  Also  in  year  two, 
after  completing  the  pilot  study  analysis  and  determining  that  the  measures  and  data  collection 
procedure  had  integrity,  we  then  initiated  the  full  study  data  collection.  The  full  study  addresses 
predictors  and  outcomes  of  the  motivational  variables  in  a  much  fuller  and  detailed  examination 
of  the  variables  than  in  the  pilot  study,  which  was  relatively  limited  and  focused  in  nature.  In  the 
full  study,  our  major  hypotheses  will  be  tested. 

Contributions  to  Basic  Science 

There  has  been  a  good  bit  of  research  on  predicting  leader  performance.  However,  there 
has  been  relatively  little  research  on  predicting  leadership  development  behavior.  For  example, 
while  motivation  to  lead  and  self-efficacy  for  leadership  have  been  explored  in  research,  self- 
efficacy  for  leadership  development  and  motivation  to  develop  leadership  skills  has  not.  Also, 
especially  given  differing  ideas  about  whether  leaders  are  bom  or  made,  there  may  be  differences 
in  how  aggressively  and  effectively  potential  leaders  pursue  the  development  of  leadership  skills. 
Successful  development  of  leadership  skills  depends  on  strong  motivation  to  develop  these  skills. 
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The  current  research  will  test  a  model  of  leadership  development  motivation  that  specifically 
targets  motivation  to  develop  leadership  skills. 

A  recent  research  article  (Ruvolo,  Petersen  &  LeBouef,  2004,  Consulting  Psychology 
Journal)  authored  by  officers  in  the  U.S.  Army  is  titled  “Leaders  Are  Made,  Not  Bom,’'  which 
strongly  reflects  the  notion  that  leadership  can  be  developed.  Further,  there  are  many  examples 
of  successful  leadership  training  and  development  programs.  However,  it  is  well  known  that 
people  differ  markedly  on  their  beliefs  that  leaders  are  'bom  or  made.’  Beliefs  that  leadership 
attributes  are  innate  or  fixed  and  are  not  acquirable  or  improvable  might  be  a  fundamental  belief 
that  can  set  the  stage  for  someone's  overall  motivation  for  leadership  development.  To  the  extent 
that  a  person  believes  that  leaders  are  bom  and  not  made,  this  can  be  a  substantial  impediment  to 
optimal  motivation  for  developing  leadership  skills.  This  research  will  explore  this  belief  and  the 
role  that  the  construct  plays  in  leadership  development  motivation.  In  addition,  the  similarities 
and  differences  between  constructs  associated  with  leadership  performance  and  leadership 
development  will  be  addressed  conceptually  and  empirically.  This  includes  the  distinction 
between  self-efficacy  for  leadership  vs.  self-efficacy  for  leadership  development,  and  motivation 
to  lead  vs.  motivation  to  develop  leadership  skills.  Differences  in  the  antecedents  and  outcomes 
of  these  constructs  also  will  be  explored  to  establish  their  distinct  construct  validity.  These  issues 
have  not  been  explored  previously  in  relation  to  leadership  development  but  do  have  great 
theoretical  and  practical  promise. 

Potential  Army/Military  Applications 

Developing  these  new  constructs  in  the  leadership  domain  and  establishing  the  construct 
validity  of  them  in  basic  research  could  lead  to  applied  research  in  key  areas  addressed  by  AR1. 
Generally,  this  applied  research  could  fall  into  two  categories:  individual  difference  predictive 
studies  and  experimental  research  in  which  manipulations  are  used.  First,  these  constructs  and 
associated  assessment  methods,  along  with  traditional  predictors  such  as  intelligence  and 
personality,  could  be  used  as  predictors  of  performance,  motivation,  development  and 
adaptability  in  Army  training  and  field  settings.  To  the  extent  that  these  measures  predict 
effectiveness  in  adapting  to  leadership  roles  and  learning  and  developing  necessary  skills,  these 
tools  might  be  used  in  selection  and  placement  within  the  Army  to  identify  those  individuals  who 
will  likely  most  benefit  from  leadership  training  and  who  are  most  likely  to  take  initiative  to 
continually  develop  and  improve  leadership  relevant  characteristics  in  themselves.  These 
individual-difference  constructs  and  assessment  tools  might  be  explored  in  applied  research  for 
relationships  with  key  behavioral  and  performance  criteria  relevant  to  leadership  and  leadership 
development  and  learning  in  practical  Army  settings. 

There  is  also  a  second  way  that  developing  these  new  constructs  in  the  leadership  domain 
in  basic  research  could  lead  to  applied  research  in  key  areas  addressed  by  ARI.  Research  might 
be  done  on  ways  of  influencing  these  constructs  in  Army  personnel  to  improve  motivation  for 
leadership  development.  Variables  such  as  self-efficacy  are  subject  to  enhancement  through 
mastery  experiences,  persuasion,  modeling  or  vicarious  learning.  Self-efficacy  for  leadership 
development  might  also  be  influenced  through  these  types  of  tactics.  For  example,  some  research 
has  manipulated  expectations  for  military  personnel  and  found  effects  for  increasing  self- 
efficacy.  Applied  research  might  be  conducted  to  determine  whether  self-efficacy  for  leadership 
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development  can  be  influenced,  and  whether  this  enhances  motivation  for  development  of 
leadership  skills  and  subsequent  development  and  performance.  Prior  research  and  theory 
suggests  that  this  should  be  possible. 

It  also  should  be  possible  to  conduct  applied  research  on  ways  in  which  beliefs  about 
improvability  of  leadership  skills  can  be  influenced.  Prior  research  in  settings  not  involving 
leadership  and  leadership  development  suggests  that  implicit  theories  of  ability  (beliefs  about 
improvability  of  attributes)  are  susceptible  to  influence.  It  should  be  possible  to  determine  if 
influences  on  beliefs  about  improvability  of  leadership  skills  have  a  subsequent  influence  on 
motivation  to  develop  leadership  skills  and  attributes  as  people  increasingly  believe  that  it  is 
possible  to  develop  and  improve  specific  characteristics.  Thus,  to  the  extent  that  these  constructs 
are  developed  and  validated  in  this  basic  research,  additional  applied  research  could  follow. 

Future  Plans 

In  year  three,  the  full  data  collection  will  proceed,  and  when  complete,  the  full  data 
analysis  and  interpretation  will  be  done  based  on  all  data  (time  1  and  time  2)  as  described 
previously.  A  full  research  report  of  the  findings  along  with  measures  and  psychometric 
properties  will  be  the  outcome.  Publication  of  this  research  in  (a)  scientific  outlet(s)  will  be 
pursued. 
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Identifying  Individual  Attributes  and  Learning  Principles 
that  Foster  Adaptive  Performance  and  Promote  Rapid  Adaptability  Skill 
Acquisition  Through  Multiple  Modes  of  Development 

Contract  Number:  W74V8H-05-K-0004  Contract  Dates:  01/15/2005  to  07/31/2007 

Institution:  George  Mason  University  PI:  Stephen  J.  Zaccaro 

Liaison:  Robert  Pleban,  IFRU 

Problem(s)/Research  Question(s)  -  What  training  and  development  principles  produce  the 
most  effective  acquisition  and  growth  of  leader  adaptability  skills?  What  individual  attributes 
moderate  the  impact  of  training  and  development  interventions  on  adaptability  skills,  and  what 
combinations  of  leader  attributes  and  team  processes  lead  to  adaptive  performance  in  today’s 
military  operational  environments? 

Technical  Barrier(s)  -  Past  theories  and  models  of  leadership  and  team  training  are  inadequate 
for  understanding  how  to  develop  adaptive  leaders  and  teams.  Training  and  developing 
adaptability  skills  for  both  individual  Soldiers  and  teams  requires  learning  strategies  that  are 
different  from  those  traditionally  employed  by  the  Army. 

Significance/Impact  for  Basic  Research  -  The  successful  completion  of  this  research  effort 
should  contribute  significantly  to  the  literature  on  leader  and  team  development,  in  particular  on 
strategies  for  growing  individual  and  team  level  adaptability  skills  through  formal  instruction, 
developmental  work  assignments,  and  self-development.  We  should  also  add  new  information  to 
the  small  but  growing  literature  on  leader  attributes  patterns  and  their  influence  on  leader 
adaptive  performance 

Potential  Transitions  -  The  products  of  this  research  should  provide  the  basis  for  applied 
research  and  application  in  various  Army  leadership  training  and  development  programs,  such  as 
those  being  conducted  at  Fort  Benning  and  Fort  Leavenworth.  Other  specific  applications  that 
can  be  explored  through  future  potential  DoD  SBIRs  may  include: 

•  The  development  of  formal  training  programs  that  target  leader  adaptability  skills;  and 

•  The  refinement  and  validation  of  assessment  tools  that  can  be  used  in  Army  training  and 
development  activities  to  measure  developmental  work  experiences,  learning  attributes, 
and  adaptive  performance  processes. 

Overview 

The  overall  purposes  of  this  research  effort  are  to  expand  current  understanding  about  the 
nature  of  adaptive  leadership,  and  to  explore  the  most  effective  ways  of  developing  adaptability 
skills  through  three  modes  of  training  -  formal  instruction,  operational  assignments,  and  self 
development.  We  build  on  prior  research  to  explore  the  leader  attributes  that  promote 
effectiveness  as  environment  conditions  become  more  dynamic  and  adversarial.  The 
specification  of  adaptive  leader  processes  and  attributes  sets  the  stage  for  a  consideration  of  best 
practices  in  leader  training  and  development  to  foster  growth  in  leader  adaptability.  Accordingly, 
in  this  research  effort  we  investigate  training  and  development  principles  that  are  likely  to 
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produce  the  most  effective  growth  of  particular  leader  adaptability  skills.  We  also  examine  how 
these  principles  vary  across  different  modes  of  Army  leader  development. 

Prior  theory  and  empirical  research  has  supported  the  premise  that  certain  forms  of 
leadership  communications,  in  the  form  of  sense-giving  and  performance  feedback,  specifically 
process-oriented  feedback,  contributes  to  growth  in  adaptive  performance.  We  are  extending  this 
research  to  examine  the  influences  of  a  training  design  that  incorporates  (a)  a  problem  centered 
approach  that  utilizes  multiple  and  qualitatively  different  performance  practice  scenarios,  and  (b) 
strategies  for  enhancing  the  practice  of  metacognitive  thinking  and  performance  regulation  skills. 
The  overall  purpose  of  these  training  strategies  is  to  enhance  the  degree  to  which  trainees  engage 
in  greater  frame-switching  processes  when  operating  environments  become  more  dynamic.  We 
also  are  examining  and  extending  these  training  principles  in  the  context  of  different  training 
approaches,  including  work  experiences  and  self-learning  strategies. 

This  research  effort  also  includes  an  examination  of  leader  attributes  that  contribute  to 
effective  adaptive  leadership.  These  personal  attributes  include  such  qualities  as  cognitive 
flexibility,  self  regulation  and  metacognitive  skills,  social  and  emotional  intelligence,  optimism, 
tolerance  for  ambiguity,  and  resilience.  Relatively  little  research  has  examined  how  these 
attributes  are  integrated  and  how  such  integrations  are  linked  to  the  processes  of  leader 
adaptation. 

Research  Approach 

We  have  recently  completed  the  second  year  of  the  project.  In  year  one  of  this  effort,  we 
constructed  an  annotated  bibliography  of  studies  on  adaptability.  We  found  74  studies  to  date 
and  coded  them  for  the  type  of  method  use,  and  the  particular  focus  of  analysis  (adaptive 
performance,  adaptability  processes,  adaptability  attributes,  training  and  development,  and 
assessment  tools).  We  also  coded  for  level  of  analysis  (individual,  leader,  and  team).  We  found 
representative  studies  in  all  of  our  coding  categories,  with  the  fewest  studies  (10)  on  the 
assessment  of  adaptability.  We  also  found  that  additional  research  was  most  needed  on: 

•  Predictive  validation  of  linkages  in  models  of  adaptability  attributes,  processes  and 
performance; 

•  Validation  of  assessment  tools;  and 

•  Empirical  examinations  of  training  designs  that  focus  specifically  on  the  development  of 
adaptability  skills. 

We  are  currently  updating  the  bibliography  and  preparing  a  conceptual  paper 
summarizing  our  full  results.  In  the  first  year  of  this  effort,  we  also  completed  an  experiment  that 
examined  the  effects  of  variability  in  training  practice  scenarios  on  adaptive  performance  in 
teams.  Adaptability  typically  requires  a  process  of  "frame-switching"  that  results  in  new  and 
different  ways  of  examining  problems.  Accordingly,  adaptability  training  should  provide 
opportunities  to  practice  such  frame-switching.  In  this  experiment,  teams  completed  a  decision¬ 
making  task  (SimCity4),  in  which  they  practiced  solving  a  series  of  three  problems.  In  one 
condition,  groups  received  a  series  of  problems  in  which  all  three  required  similar  decision 
strategies.  In  another  condition,  groups  received  a  series  of  problems  in  which  each  required  a 
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qualitatively  different  decision  strategy.  Then  all  groups  were  given  an  adaptive  performance 
trial  that  required  decision  strategies  different  from  all  previous  trials.  Groups  were  free  to  make 
as  many  decisions  as  necessary  to  solve  the  key  issues  in  the  adaptive  performance  trial.  Possible 
decision  solutions  in  this  last  trial  ranged  from  low  to  high  in  their  requirements  for  frame¬ 
switching.  This  research  approach,  then,  tests  the  proposed  association  between  one  key  training 
parameter,  practice  scenario  variability,  and  team  adaptability. 

In  year  two  of  the  research  effort  we  completed  another  experiment  (described  below), 
initiated  a  third,  and  initiated  several  field  studies.  These  are  described  in  more  detail  below. 

Accomplishments 

The  data  from  Investigation  1  indicated  support  for  the  proposed  hypothesis.  As  Figure  1 
shows,  the  variable  scenario  practice  condition  resulted  in  greater  numbers  of  overall  decisions 
than  the  routine  practice  decisions,  but  the  largest  differences  were  on  decisions  reflecting 
greater  frame-switching.  The  frequency  of  high  frame-switching  decisions  was  also  significantly 
correlated  with  a  key  parameter  of  overall  adaptive  performance. 


%  of  groups 
making  decision 


I  I  —  - 

Routine  Groups  Variable  Groups 


I  I  Low  frame  switching 
I  1  Medium  frame  switching 
■  High  frame  switching 


N  =  30. 


Figure  1.  Decisions  made  by  groups. 


Note :  Groups  make  multiple  decisions  during  the  adaptive  performance  test  trial.  These  decisions  vary  in 
terms  of  whether  they  reflect  low,  medium,  or  high  frame-switching.  This  figure  shows  that  the  largest 
differences  between  the  groups  were  in  the  number  of  high  frame-switching  decisions  made  by  group 
members. 


We  assessed  several  potential  mediators  to  explain  the  effects  of  training  practice 
variability  on  adaptive  performance,  including  collective  efficacy  for  dealing  with  change,  and 
the  accuracy  and  sharedness  of  mental  models.  Groups  did  not  differ  significantly  on  these 
variables,  suggesting  that  the  primary  consequence  of  practice  variability  is  to  prompt  the 
occurrence  of  more  frame-switching  processes  during  adaptation. 
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In  the  second  year  of  this  effort,  we  completed  a  second  experiment  that  examined  the 
role  of  adaptive  self-regulation  training  in  fostering  effective  frame-switching.  In  Investigation  1, 
all  groups  had  received  such  training  before  beginning  the  practice  trials.  In  Investigation  2,  all 
groups  experienced  variability  in  practice  scenarios,  but  only  half  of  them  received  adaptive  self¬ 
regulation  training.  In  this  study  we  found  a  pattern  similar  to  Study  1  -  groups  receiving 
qualitative  variability  in  practice  scenarios  showed  greater  adaptability  in  the  test  trial  than 
groups  not  receiving  such  variability.  This  effect  was  somewhat  moderated  by  the  receipt  of  self¬ 
regulation  training.  Groups  that  received  variable  scenarios  but  no  self-regulation  training  did  not 
tend  to  exhibit  the  same  degree  of  adaptability  in  the  test  trial. 

Because  this  moderating  effect  was  modest  in  size,  we  are  completing  two  more  studies, 
one  with  college  students  and  the  other  with  Soldiers  at  Fort  Benning.  Our  focus  is  on  sharpening 
the  self-regulation  training  and  examining  in  more  detail  the  changes  occurring  in  individual 
cognitive  processes  as  these  individuals  encounter  the  variable  scenarios  under  high  and  low 
conditions  of  adaptive  self-regulation  training.  The  results  of  these  studies  should  shed  some 
light  on  the  role  of  both  training  parameters  on  cognitive  processes  linked  to  adaptive 
performance.  We  also  are  examining  the  role  of  individual  differences  in  cognitive  flexibility  and 
metacognitive  skills  in  moderating  the  proposed  effects  of  training  parameters. 

In  the  second  year  of  the  project  we  also  ( 1 )  completed  construction  of  a  survey  that 
assesses  qualities  of  work  experiences  proposed  as  contributing  to  growth  in  individual 
adaptability  skills,  and  (2)  completed  the  development  of  tools  that  can  be  used  in  self-learning 
programs  focusing  on  leadership  adaptability  skills.  We  also  developed  a  constructed-response 
assessment  of  adaptive  decision  making  in  complex  and  dynamic  business  scenarios.  These 
instruments  incorporate  the  basic  principles  of  growing  adaptability  skills  that  are  being 
examined  in  the  laboratory  studies.  We  are  currently  administering  these  instruments  to 
managers  in  a  variety  of  business  settings  to  test  the  validity  of  these  learning  principles,  as 
operationalized  in  developmental  work  assignments  and  self-development  exercises.  As  part  of 
these  field  research  efforts,  we  also  are  administering  measures  of  individual  differences 
proposed  as  being  associated  with  adaptability. 

Contributions  to  Basic  Science 

This  research  effort  should  provide  substantial  benefits  by  enhancing  our  understanding 
of  how  to  train  and  develop  adaptability  skills  through  formal  course  instruction,  as  well  as 
through  officer  assignments  and  self-development  efforts.  This  research  effort  also  should 
contribute  to  the  growing  literature  on  how  different  characteristics  of  leaders  are  linked  in 
patterns  to  influence  leader  adaptive  performance,  as  well  as  the  leader's  responsiveness  to 
different  training  interventions. 

Potential  Army/Military  Applications 

Some  of  the  anticipated  products  of  this  research  effort  include  training-based  assessment 
tools  and  training  protocols  that  can  contribute  to  more  effective  development  of  Soldiers  and 
offices.  Several  training  units,  including  those  at  Ft.  Benning  and  Ft.  Leavenworth,  are  interested 
particularly  in  developing  more  adaptive  leaders.  We  are  using  a  computer-based  training 
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simulation  at  Fort  Benning  to  test  our  hypotheses.  We  have  completed  an  initial  round  of  data 
collection,  and  have  been  invited  back  to  collect  additional  data.  If  our  hypotheses  are  confirmed, 
our  experimental  design  and  reprogramming  of  some  of  the  scenarios  in  the  simulation  have  the 
potential  of  providing  some  direct  applications  for  adaptability  training  to  the  BOLC  programs 
offered  at  Fort  Benning. 

Future  Plans 

We  expect  in  the  final  year  of  this  research  effort  to: 

•  Complete  data  collection  and  analysis  of  our  third  laboratory  investigation  with 
undergraduate  students; 

•  Complete  data  collection  and  analysis  for  our  investigation  at  Fort  Benning; 

•  Complete  data  collection  and  analysis  of  surveys  to  business  managers  that  measure  the 
quality  of  leader  work  experiences; 

•  Complete  data  collection  and  analysis  on  the  self-development  study  administered  to 
direct  marketing  and  sales  managers;  and 

•  Complete  all  data  analyses  and  prepare  research  reports  on  each  study,  as  well  as  a  paper 
on  the  conceptual  foundations  of  this  research  effort. 
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BRU  RESEARCH  OBJECTIVE  #3:  PROVIDE  FUNDAMENTAL 
KNOWLEDGE  FOR  IDENTIFYING,  ASSESSING,  AND  ASSIGNING 
QUALITY  PERSONNEL  FOR  THE  EVOLVING  ARMY 


Research  in  this  section  is  directed  toward  identifying  and  measuring 
the  aptitudes  and  skills  that  are  unique  to  the  human  performance 
requirements  of  the  Future  Force.  Exploring  the  sociological  and 
psychological  factors  that  could  influence  recruitment,  retention,  and  Army 
performance  are  important  aspects  of  this  research  objective. 
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An  Interactionalist  Analysis  of  Soldier  Retention 
Across  Career  Stages  and  Time 

Contract  #:  W74V8H-04-K-0002  Contract  Dates:  06/01/2004  to  09/30/2005 

Institution:  Texas  A&M  University  PI:  Gilad  Chen 

Co-PI:  Robert  Ployhart 
Liaison:  Michael  Rumsey,  SARU 

Problem(s)/Research  Question(s)  -  How  do  personal,  situational,  and  temporal  factors  help 
explain  retention  processes  at  different  career  stages? 

Technical  Barrier(s)  -  Models  of  retention  have  yet  to  consider  dynamic  processes  leading  to 
turnover,  and  whether  retention  processes  might  differ  at  different  career  stages. 

Significance/Impact  for  Basic  Research  -  Understanding  how  and  why  the  decision  to  stay  or 
leave  an  organization  unfolds  over  time,  and  whether  personal  and  situational  factors 
differentially  drive  these  processes  at  different  career  stages. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  RETAIN:  Strategies  to  Retain  Quality  Soldiers  (WP  271 ); 

•  Soldier  Attitudes  and  Opinions  in  a  Changing  Army  (WP  102);  and 

•  SELECT21 :  Selection,  Classification,  and  Performance  Metrics  for  the  Objective  Force 
Soldier  (WP  257)  and  ATO:  IV.HS.2002.01. 

Overview 

As  a  result  of  both  the  collapse  of  the  former  Soviet  Union  and  the  horrendous  events  of 
September  11,  2001,  the  U.S.  has  engaged  in  a  series  of  conflicts  around  the  globe,  oftentimes 
simultaneously.  In  today’s  reality,  and  probably  even  more  so  in  the  future,  the  U.S.  armed 
forces  must  be  flexible  and  adaptable  in  their  ability  to  deploy  the  same  troops  to  qualitatively 
different  types  of  conflicts,  with  vastly  different  purposes  (e.g.,  dismantling  terror  and/or  drug 
networks,  homeland  defense,  peace  keeping)  and  in  different  geographical  locations. 
Consequently,  the  U.S.  armed  forces  have  put  a  high  premium  on  selecting,  training,  and 
maintaining  the  sufficient  human  resources  required  for  being  competitive  in  this  new  global 
reality.  Furthermore,  effective  retention  has  never  been  as  consequential  as  it  is  today,  given  the 
high  costs  involved  in  attracting,  selecting,  and  training  Soldiers. 

Unfortunately,  current  theoretical  models  of  retention  provide  an  insufficient  basis  for  our 
understanding  of  retention  in  such  complex  and  dynamic  work  environments.  Our  research 
addresses  three  limitations  of  previous  retention  research: 

1 .  Organizations  can  enhance  retention  through  management  practices  directed  at  creating 
more  positive  work  situations,  and  through  staffing  practices  directed  at  selecting 
employees  who  would  better  fit  the  work  environment.  Accordingly,  the  first  contribution 
of  the  present  research  is  the  simultaneous  examination  of  both  person  variables 
(cognitive  ability)  and  situational  variables  (social  support  and  work  challenge)  as 
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predictors  of  the  retention  process.  In  particular,  we  propose  that  lower  levels  of  ability 
and  higher  levels  of  social  support  and  work  challenge  are  associated  with  more  positive 
levels  of job  attitudes,  and  thus  lower  likelihood  of  turnover.  We  further  consider  possible 
person-by-situation  interactions,  and  propose  that  the  relationships  between  ability  and 
job  attitudes  are  less  negative  when  social  support  and  work  challenge  are  higher. 

2.  There  is  now  a  growing  awareness  that  factors  that  affect  work  experiences  may  vary 
across  career  stages,  given  that  employees  require  and  desire  different  things  at  different 
stages  of  their  careers.  In  the  context  of  the  U.S.  Army,  for  instance,  there  could  be 
different  reasons  why  first  and  second  term  Soldiers  decide  to  re-enlist.  At  early  career 
stages.  Soldiers  are  adjusting  to  their  new  military  career,  and  thus  social  support  is 
particularly  critical.  However,  at  later  career  stages,  Soldiers  are  more  likely  to  appreciate 
and  benefit  from  challenging  work  assignments.  Accordingly,  a  second  contribution  of 
our  research  is  the  examination  of  whether  retention  processes  are  affected  by  different 
factors  at  different  career  stages.  In  particular,  we  propose  that  cognitive  ability  and 
social  support  are  more  likely  to  relate  to  job  attitudes  and  retention  at  earlier  career 
stages,  whereas  work  challenge  is  more  likely  to  relate  to  job  attitudes  and  retention  at 
later  career  stages. 

3.  Finally,  given  that  employees  in  many  organizations  (including  the  U.S.  Army)  transition 
from  one  work  environment  or  project  to  another  quite  frequently,  it  is  critical  to  know 
whether  such  transitions  are  associated  with  changes  in  retention  decisions  as  well  as  in 
proximal  drivers  of  retention  outcomes  (e.g.,  job  attitudes),  and  whether  capturing 
particular  patterns  of  change  (i.e.,  growth/increase  or  decline/decrease)  in  variables 
capturing  the  retention  process  can  help  explain  additional  variance  in  turnover.  In  our 
research,  we  propose  that  more  positive  changes  in  job  attitudes  are  associated  with 
decreased  likelihood  of  turnover.  Moreover,  we  argue  that  changes  in  job  attitudes 
capture  a  unique  phenomenon  different  from  average  levels  of  job  attitudes,  and  thus 
changes  in  job  attitudes  should  account  for  additional  variance  in  retention  decisions  over 
and  above  average  levels  of  job  attitudes. 

Research  Approach 

We  tested  our  propositions  using  field  survey  data  collected  by  Walter  Reed  researchers 
(called  the  OPTEMPO  project)  from  over  one  thousand  Soldiers  at  multiple  times  over  the 
course  of  two  years.  The  survey  data  were  linked  to  Soldiers'  ability  scores,  which  were 
collected  during  the  selection  process  into  the  U.S.  Army.  The  surveys  captured  multiple  aspects 
of  the  work  environment,  including  social  support  (i.e.,  perceived  quality  of  officer  and  non- 
commisioned  officer  leadership,  co-worker  support)  and  work  challenge  (i.e.,  job  challenge, 
significance  of  work  tasks,  and  whether  Soldiers  were  in  deployment,  in  training,  or  in  their 
garrison  base).  At  each  time  period.  Soldiers  also  indicated  their  job  attitudes  (i.e.,  job 
satisfaction,  job  involvement,  and  general  morale)  and  intention  to  re-enlist  with  the  Army. 
Finally,  career  stage  data  also  are  available.  These  rich  data  are  unique  in  that  they  allow  us  to 
capture  and  explain  retention  processes  as  they  unfold  over  time,  and  at  different  career  stages. 
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Accomplishments 

Results  provided  mixed  support  for  the  propositions.  In  particular: 

1 .  First,  although  cognitive  ability  did  not  predict  job  attitudes  or  turnover  intentions, 
situational  variables  (social  support  and  work  challenge)  strongly  predicted  job  attitudes, 
which  in  turn  predicted  turnover  intentions.  However,  no  interactions  between  ability  and 
situational  variables  were  detected.  Thus,  the  results  supported  the  relationships  between 
situational  variables  and  turnover  intentions,  as  mediated  by  job  attitudes,  but  cognitive 
ability  played  no  role  in  these  relationships. 

2.  Second,  the  data  did  not  support  our  career  stage  propositions.  Simply  put,  the  results  did 
not  differ  for  Soldiers  at  different  career  stages,  as  measured  by  military  tenure  and  rank. 

3.  Finally,  our  proposition  that  more  positive  changes  in  job  attitudes  are  associated  with 
decreased  likelihood  of  turnover  was  largely  supported.  In  particular,  we  found  that 
Soldiers  differed  in  the  extent  to  which  their  job  attitudes  improved  or  got  worse  over 
time,  and  that  those  whose  job  attitudes  improved  more  over  time  also  indicated  they  are 
more  interested  to  re-enlist  with  the  U.S.  Army  for  additional  terms. 

Contributions  to  Basic  Research 

This  project  contributes  to  the  employee  retention  literature  in  several  important  ways. 
First,  by  integrating  person  and  situation  theories  of  employees  retention,  this  research  provided 
a  more  comprehensive  picture  of  the  retention  process  and  its  determinants.  Second,  this  research 
suggested  that  drivers  of  retention  do  not  seem  to  differ  across  career  stages,  and  thus  practices 
and  principles  for  retaining  employees  are  likely  to  generalize  across  career  stages.  Finally, 
moving  beyond  the  mostly  static  retention  research,  this  research  provided  a  richer,  more 
dynamic  view  of  how  and  why  employees'  decisions  to  stay  or  leave  the  organization  evolve 
over  time  vis-a-vis  their  experiences  at  work.  In  particular,  we  showed  that  employees'  decision 
to  stay  or  leave  the  Army  change  over  time,  and  corresponds  to  whether  their  job  attitudes 
improve  or  get  worse  over  time. 

Potential  Army/Military  Applications 

This  research  has  the  potential  to  advance  Army/military  applications  in  several  ways. 
First,  given  that  U.S.  armed  forces'  selection  systems  rely  heavily  on  cognitive  ability,  it  is 
important  to  ensure  that  more-qualified  Soldiers  (i.e.,  those  higher  in  ability)  are  those  remaining 
in  the  military.  Failing  to  find  reliable  relationship  between  cognitive  ability  and  the  retention 
process  is,  in  a  way,  good  news;  it  suggests  selecting  Soldiers  based  on  cognitive  ability  will  not 
be  likely  to  affect  retention  rates  (while,  at  the  same  time,  help  promote  the  high  level  of 
performance  associated  with  ability).  Second,  our  results  that  predictors  of  turnover,  particularly 
job  attitudes  and  situational  variables,  do  not  differ  at  different  career  stages  suggest  that  the 
same  practices  directed  at  retaining  Soldiers  can  be  effective  at  different  career  stages.  Finally, 
given  that  this  investigation  tracked  Soldiers  over  the  course  of  2  years  as  they  transition  across 
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different  work  assignments  (i.e.,  training  or  deployment),  it  helped  to  better  explain  the  decision 
process  associated  with  retention. 

In  particular,  our  research  showed  that,  over  time.  Soldiers'  experiences  and  attitudes  are 
shaped  by  social  support  and  work  challenge,  and,  as  these  experiences  and  attitudes  improve  or 
get  worse  over  time,  so  does  the  likelihood  Soldiers  would  leave  or  stay  with  the  military.  As 
such,  our  research  suggests  that  leaders  need  to  continuously  support  and  assign  challenging  and 
interesting  work  tasks  to  their  Soldiers,  as  doing  so  would  likely  help  improve  their  job  attitudes 
and  thus  increase  the  likelihood  of  retention.  Altogether,  this  research  will  likely  help  the 
military  improve  various  management  programs  directed  at  enhancing  retention,  and  help  align 
these  programs  with  the  military’s  staffing  strategies. 

Final  Summary 

The  results  of  our  study  have  several  implications  for  how  staffing  and  managerial 
practices  relate  to  Soldiers'  decisions  to  reenlist  with  the  U.S.  Army.  First,  Soldiers'  cognitive 
ability  did  not  relate  to  Soldiers'  job  attitudes  (work-related  morale,  satisfaction,  and 
psychological  involvement)  and  reenlistment  intentions.  These  findings  are  promising  in  as  much 
as  they  suggest  that  reliance  on  ability  testing  in  selecting  recruits  into  the  Army  would  unlikely 
in  and  of  itself  lead  to  lower  levels  of  morale,  satisfaction  and  reenlistment  rates.  Second,  we 
found  that  social  support  and  work  challenge  variables  positively  promoted  job  attitudes,  and  that 
job  attitudes,  in  turn,  positively  related  to  reenlistment  intentions.  These  results  suggest  that 
leadership  and  work  design  practices  that  enhance  social  support  and  job  challenge  can  motivate 
Soldiers  to  reenlist  with  the  Army,  because  they  promote  more  positive  work  attitudes.  That  is, 
leaders  should  motivate  Soldiers  by  simultaneously  challenging  and  supporting  them.  However, 
our  final  set  of  findings  suggest  that,  the  extent  to  which  job  attitudes  improve  over  time  also 
plays  a  role  in  whether  or  not  Soldiers  intend  to  reenlist.  These  results  imply  that  leaders  should 
ensure  they  continuously  strive  to  improve  Soldiers'  morale,  satisfaction,  and  involvement,  to 
enhance  chances  of  Soldier  reenlistment. 
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Problem(s)/Research  Question(s)  -  Is  time-constrained  performance  within  a  given  cognitive 
task  domain  determined  by  one  ability,  called  level  of  performance,  or  by  at  least  two  abilities, 
level  of  performance  and  speed  of  performance? 

Technical  Barriers  -  (a)  Undeveloped  theory  of  the  cognitive  abilities  responsible  for  time- 
constrained  performance  of  tasks  having  non-trivial  difficulty;  and  (b)  the  absence  of  valid 
models  and  procedures  to  measure  speed  of  performance  of  cognitive  tasks  having  non-trivial 
difficulty. 

Significance/Impact  for  Basic  Research  -  If  our  project  is  successful,  we  will  report  the  first 
psychometrically  valid  method  for  measuring  average  speed  of  performance  of  cognitive  tasks 
having  non-trivial  difficulty.  Moreover,  the  most  widely  accepted  model  of  the  structure  of 
cognitive  abilities  (Carroll,  1993)  will  need  to  be  revised. 

Potential  Transitions  -  This  research  could  potentially  lead  to: 

•  Improved  measures  of  key  attributes  that  Soldiers  need  for  effective  performance  in 
assignments  for  which  mental  quickness  and  accuracy  are  important; 

•  Enhanced  predictive  validity  of  the  Armed  Forces  Qualification  Test  (AFQT),  if  that  test 
were  modified  to  measure  performance  speeds  in  each  of  the  cognitive  domains  it 
assesses; 

•  Performance  scores  that  can  be  used  in  a  model  that  matches  persons  to  jobs  (i.e., 
improved  methods  for  classifying  and  assigning  force  members);  and 

•  Improved  training  programs  in  areas  where  mental  quickness  and  accuracy  are  critical. 


Overview 


Few  would  doubt  that  if  each  person  in  a  group  were  assigned  a  set  of  non-trivial 
cognitive  tasks  of  like  kind,  then  people  would  vary  in  how  successfully  and  rapidly  they 
performed  those  tasks.  Psychologists  have  known  for  several  decades  how  to  measure  validly  a 
person's  ability  to  successfully  perform  a  set  of  non-trivial  tasks  from  a  narrow  cognitive  domain 
(e.g.,  spatial  visualization  and  inductive  reasoning) — provided  task  performance  is  self-paced. 
Relative  to  a  given  cognitive  task  domain,  that  ability  is  called  level. 

On  the  other  hand,  there  currently  does  not  exist  a  valid  way  to  measure  how  quickly  a 
person  successfully  performs  a  set  of  non-trivial  cognitive  tasks  when  strict  time-limits  for  task 
completion  are  imposed.  Few  psychometricians  appear  to  be  aware  of  this  methodological  gap, 
or  even  aware  that  in  a  given  cognitive  domain  a  quickness  or  speed  ability  might  exist  and  be 
measurable.  In  part  this  may  be  because  none  of  the  several  dozen  (rarely  cited)  studies 
conducted  in  the  20th  century  to  measure  domain-specific  speed  abilities  were  successful. 
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Those  previous  attempts  to  measure  speed  abilities  in  specific  cognitive  domains  failed 
primarily  because  investigators  used  faulty  measurement  models.  For  example,  many 
investigators  employed  speed  scores  that  were  some  function  of  a  person's  collective  item 
response  times  (e.g.,  mean,  median,  or  total).  That  approach,  however,  ignores  the  likely 
influences  that  varying  item  characteristics  have  on  item  response  times,  much  as  they  do  on 
binary  item  scores  (right  and  wrong  scores).  Some  researchers  who  used  this  approach  also  used 
both  right  and  wrong  answer  times  when  computing  speed  scores,  and  thereby  assigned  speed 
scores  confounded  by  extraneous  variables  such  as  time  taken  to  guess  answers  and  time  taken  to 
consider  and  then  abandon  items.  Other  researchers  tried  to  adapt  classical  test  theory's  true-and- 
error  score  model  to  the  measurement  of  speed.  But  that  model — which  was  the  dominant 
psychometric  model  for  most  of  the  20Ih  century — was,  a  priori ,  ill-suited  for  measuring 
hypothetical  speed  ability.  The  true-and-error  score  model  postulates  that  a  person's  success  or 
failure  on  a  cognitive  test  item,  regardless  of  the  conditions  of  performance,  is  a  function  of  only 
one  ability,  the  person's  so-called  true  score  in  the  given  cognitive  domain. 

In  this  study  we  will  use  modem  measurement  models  in  an  attempt  to  measure  an 
average  speed  ability  in  the  domain  of  numerical  reasoning.  More  formally,  we  will  test  what  we 
call  the  speed-level  hypothesis',  in  narrow  cognitive  task  domains,  two  abilities — speed  and 
level — determine  a  person's  success  or  failure  in  time-constrained  task  performance.  Our  null 
hypothesis  is  that  level  ability  alone  determines  a  person's  success  or  failure  in  both  time- 
constrained  and  self-paced  task  performance. 

Research  Approach 

We  are  conducting  a  laboratory  study  wherein  we  have  tested  a  large  sample  of 
University  of  Minnesota  college  students  under  two  performance  conditions,  self-paced  and 
time-constrained  ("speeded"). 

Using  numerical  reasoning  test  items  drawn  from  old,  published  editions  of  the  SAT  and 
GRE  general  tests,  we  constructed  two  equivalent  test  forms.  Students  were  randomly  assigned 
to  take  one  or  the  other  form  in  a  self-paced  test  session,  and  then  returned  one  or  more  days  later 
and  took  the  alternate  test  form  in  a  speeded  test  session.  This  procedure  in  effect  generated  two, 
independent  samples  of  students.  In  the  speeded  session,  each  test  item  bore  an  assigned  time¬ 
limit.  Each  item's  time-limit  was  set  to  the  median  correct  response  time  computed  for  the  given 
item  using  the  self-paced  test  data  of  a  large  initial  sample  of  students.  In  both  the  self-paced  test 
session  and  the  speeded  test  session,  test  items  were  administered  one  at  a  time  using  a  personal 
computer.  Each  student's  test  sessions  were  conducted  individually  in  the  seclusion  of  a 
relatively  sound-proofroom.  During  each  session  computers  recorded  the  student's  answers  and 
answer  times. 

To  each  of  the  two  samples'  self-paced  test  data  we  will  fit  a  measurement  model  that 
postulates  that  a  single  ability,  level,  accounts  for  students'  binary  item  scores  (right  or  wrong 
scores).  The  fitted  models  will  furnish  us  an  estimate  of  each  student's  numerical  reasoning  level 
ability. 
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The  centerpiece  of  our  data  analysis  consists  of  our  comparing  the  fit  of  two  competing 
measurement  models  to  our  subjects’  item  scores  earned  during  the  speeded  test  sessions.  One  of 
the  models  will  have  the  same  structure  as  the  one  fitted  to  the  self-paced  data,  and  will 
incorporate  each  student’s  already-estimated  level  ability.  The  other  measurement  model 
postulates  that  two  abilities — level  and  average  speed — account  for  students’  speeded  item 
scores.  We  will  confirm  our  speed-level  hypothesis  if  we  find  that  the  two-ability  model  provides 
the  better  fit  to  the  item  scores  earned  during  the  speeded  test  sessions. 

Accomplishments 

Based  on  our  data  analyses  to  date,  the  evidence  for  the  speed-level  hypothesis  is  mixed. 
We  have  found  that  in  only  one  of  our  two  subject  samples  are  item  accuracy  scores  earned 
during  speeded  testing  unambiguously  a  function  of  two  person  attributes,  one  which  is  level 
ability,  and  the  other  which  almost  certainly  is  an  average  speed  ability.  Apparently  our  rule  for 
setting  item  time-limits — for  a  given  item,  the  median  correct  response  time  computed  using  the 
self-paced  test  data  of  an  independent  sample  of  students — resulted  in  time-limits  for  one  of  our 
samples  that  were  too  generous  to  elicit  a  detectable  average  speed  ability.  To  measure  an 
average  speed  ability  in  numerical  reasoning  by  modeling  the  binary  item  accuracy  scores  of 
persons  who  vary  widely  in  level  ability,  one  should  set  at  least  some  item  time  limits  very  low — 
perhaps  as  low  as  the  10th  percentile  of  the  self-paced  correct  response  times  recorded  for  the 
item  using  an  independent  subject  sample. 

Modeling  correct  answer  times,  rather  than  binary  item  scores,  may  prove  to  be  a  more 
fruitful  approach  to  measuring  speed  ability  in  a  given  cognitive  domain.  Because  answer  times 
are  measured  on  a  continuous  rather  than  discrete  scale,  and  because  all  who  correctly  answer  a 
given  item  will  inevitably  differ  in  their  answer  times,  correct  answer  times  contain  far  more 
information  about  a  person’s  speed  of  performance  than  do  binary  item  scores.  Using  data  from 
one  of  our  samples,  we  have  successfully  fit  a  speed-and-level  model  to  40  students’  correct 
answer  times  to  four  relatively  easy  items  that  all  40  students  answered  correctly.  Using  the  fitted 
model,  we  assigned  speed  and  level  scores  to  each  of  the  40  subjects.  Subsequently  we  found 
that  those  students'  speed  scores,  in  linear  combination  with  their  level  scores,  performed  better 
than  level  scores  alone  at  predicting  the  percentage  of  48  test  items  that  the  subjects  correctly 
answered  during  speeded  testing.  There  is,  however,  a  difficult  technical  barrier  encountered 
when  modeling  correct  answer  times.  If  some  item  time-limits  are  short,  every  subject  will  miss 
at  least  some  items  during  speeded  testing.  Speeded  testing  thus  generates  a  varying  number  of 
missing  correct  answer  times  for  each  subject  and  each  item.  Because  correct  answer  times  are 
not  missing  at  random,  in  order  to  reliably  estimate  each  subject’s  speed  ability,  one  must  first 
statistically  impute  each  subject's  missing  correct  answer  times.  The  methodology  for  such 
imputation  is  very  new,  and  there  is  no  assurance  that  a  statistically  defensible  way  to  estimate 
the  missing  correct  answer  times  can  be  found. 

To  our  surprise,  in  both  of  our  subject  samples  we  identified  about  a  dozen  students  who 
correctly  answered  a  higher  percentage  of  items  during  speeded  testing  than  during  self-paced 
testing.  Though  there  are  other  possible  explanations  for  this  phenomenon,  perhaps  for  these 
students  the  challenge  of  solving  numerical  reasoning  items  bearing  time-limits  boosted  their 
motivation  to  succeed  over  that  which  they  experienced  during  self-paced  testing.  The  existence 
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of  such  subgroups  of  people,  although  they  are  few  in  number,  suggests  that,  at  least  in  the  task 
domain  of  numerical  reasoning,  relying  solely  on  a  person's  level-ability  score  could  well  lead  to 
erroneous  conclusions  about  the  person's  actual  capabilities. 

Contributions  to  Basic  Research 

If  we  are  able  to  confirm  the  speed-level  hypothesis  in  the  numerical  reasoning  domain, 
then:  (a)  we  will  be  the  first  researchers  to  have  validly  measured  a  person's  speed  of 
performance  on  a  set  of  cognitive  tasks  having  non-trivial  difficulty;  (b)  we  will  have 
demonstrated  that,  within  a  given  task  domain,  the  conditions  of  task  performance  can  strongly 
influence  what  cognitive  abilities  are  measured;  (c)  we  will  have  found  a  method  to  more 
completely  and  accurately  summarize  a  person's  proficiency  status  within  the  domain  of 
numerical  reasoning,  and  if  so,  we  expect  that  our  approach  would  likely  generalize  to  other 
cognitive  task  domains;  and  (d)  we  will  have  presented  strong  evidence  that  the  structure  of 
human  cognitive  abilities  is  much  more  complex  than  is  portrayed  by  the  most  widely  accepted 
factor-analytic  models  (Carroll,  1993;  Gustafsson,  1984);  those  models  are  based  on  global-time- 
limit  test  scores,  which  confound  level  ability  with  any  speed  ability  that  may  exist  for  a  given 
cognitive  domain. 

Potential  Army/Military  Applications 

Quickness  and  accuracy  in  problem  solving  under  emergency  or  combat  conditions  is  a 
highly  desirable  attribute  of  any  person  serving  in  a  command  or  key  technical  role  in  the  Army. 
The  desirability  of  this  attribute  increases  in  proportion  to  the  pace  at  which  technology  increases 
the  volume  and  speed  of  communications.  However,  none  of  the  four  subtests  that  comprise  the 
Armed  Forces  Qualification  Test  (AFQT)  provides  a  measure  of  performance  speed  in  any  of  the 
domains  the  AFQT  assesses.  Rather,  all  four  parts  are  intended  to  be  tests  of  power  (Russell, 
Peterson,  Rosse,  Hatten,  McHenry.  &  Houston.  2001,  p.  77).  Having  recruits'  speed  scores  in 
addition  to  their  level  scores  could  lead  to  important  improvements  in  matching  recruits  to  job 
specializations,  particularly  in  cases  where  those  persons  would  need  to  be  able  to  perform 
complex  cognitive  tasks  not  just  successfully,  but  rapidly  as  well. 

Future  Plans 

Our  future  plans  include  the  following: 

•  Construct  validate  speed  and  level  scores  generated  by  factor  analytic  models.  In 
particular,  compare  level  and  speed  scores  estimated  by  modeling  binary  item  scores  with 
those  estimated  by  modeling  correct  answer  times. 

•  Separately  fit  a  so-called  2PL  IRT  model,  designed  to  measure  level  ability,  to  item 
accuracy  scores  obtained  from  self-paced  testing  and  then  to  item  accuracy  scores 
obtained  from  speeded  testing.  Next,  compare  the  ability  estimates  derived  from  the  two 
testing  conditions.  If  the  speed-level  hypothesis  is  correct,  then  the  two  sets  of  ability 
estimates  will  not  be  linearly  related. 
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•  Investigate  the  concurrent  validity  of  subjects’  speed  and  level  scores  by  fitting 
regression  models  to  subjects’  college  admission  ACT  Math  scores. 

•  Investigate  imputing  missing  correct  answer  times  in  both  samples’  speeded  test  data.  If 
overcoming  the  statistical  complications  entailed  by  this  effort  appear  too  time- 
consuming,  we  will  postpone  this  research  until  after  we  submit  our  Final  Report. 

•  Estimate  the  association  between  numerical  reasoning  speed  and  level  scores  with 
gender,  handedness,  and  English  as  a  second  language. 
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Oswego 


Problem(s)/Research  Question(s)  -  Under  what  conditions  are  emotional  behaviors  that  in  the 
short  term  may  appear  irrational  (particularly  retribution  in  anger  and  flight  in  fear),  rational  or 
beneficial  for  an  individual  or  his  or  her  group? 

Technical  Barrier(s)  -  Ethical  and  technical  difficulties,  along  with  prohibitive  costs,  create 
significant  impediments  to  human  subject  research  that  induces  and  observes  anger,  fear,  and 
other  emotions  in  controlled  situations. 

Significance/1  mpact  for  Basic  Research  -  The  research  provides  both  a  model  and  a  new 
explanatory  framework  to  help  us  understand  how  some  complex  emotional  behaviors  evolved 
and  how  they  can  be  beneficial  or  detrimental  in  both  individual  and  social  contexts.  This  can 
provide  predictions  for  practical,  more  focused  human  subject  research. 

Potential  Transitions  -  Knowledge  to  be  gained  in  this  basic  research  may  be  usefully 
employed  in  studies  including  but  not  limited  to: 

•  ATO  III. HS. 2004. 01  -  Accelerating  Leader  Development  (WP  269  &  285); 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  103);  and 

•  Assessing  and  Developing  Leaders  for  Special  Operations  Units  (WP  218). 


Overview 


The  goal  of  this  research  is  to  develop  a  model  of  the  minimal  conditions  for  certain 
kinds  of  emotional  behaviors — particularly  flight  in  fear  and  retribution  in  anger — to  arise  in  a 
social  group  and  to  be  beneficial  for  an  individual  or  for  that  group.  The  model  also  makes 
predictions  that  can  be  tested  in  future  empirical  research,  including  empirical  research  with 
humans. 

Emotions  often  motivate  behaviors  that  are  essential  for  our  survival,  but  sometimes  they 
can  lead  to  behaviors  that  appear  irrational.  Some  of  these  seemingly  irrational  behaviors  may  in 
fact  be  irrational,  but  others  may  be  behaviors  that  on  balance  benefit  us  if  we  understand  them 
in  the  right  context.  We  have  as  of  yet  little  ability  to  distinguish  these  cases,  or  when  they  are 
likely  to  happen.  Important  tools  for  predicting  and  evaluating  the  decision  making  of  individuals 
have  been  developed  in  economics  and  related  fields  of  inquiry.  In  general,  these  tools  are  based 
on  the  assumption  that  individuals  are  rational  agents  who  evaluate  the  expected  outcome  of  their 
available  choices,  and  then  make  choices  that  lead  to  their  most  immediate  expected  benefit. 
However,  actual  empirical  research  into  human  decision  making  reveals  that  there  are  common 
and  consistently-occurring  forms  of  behavior  inconsistent  with  a  simple  interpretation  of  these 
kinds  of  theories. 
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An  early  and  surprising  result  of  experimental  economics  was  the  discovery  of  an 
unpredicted  and  seemingly  irrational  behavior  in  Ultimatum  Games  that  appears  to  be  caused  by 
anger  or  another  retributive  affect  (Roth,  1995).  An  ultimatum  game  is  a  situation  in  which  one 
individual  (the  “proposer”)  is  given  some  utility  (typically  money)  and  that  agent  must  decide 
how  to  share  this  utility  with  another  agent  (the  “receiver”).  The  receiver  has  no  say  in  the 
distribution,  but  can  decide  to  “veto”  the  offer,  resulting  in  neither  individual  getting  anything. 
Traditional  economic  and  game  theoretic  models  predict  that  the  receiver  will  never  exercise 
veto,  but  will  rather  accept  any  offer  greater  than  zero,  since  something  is  better  than  nothing; 
and  predict  that  the  proposer  agent,  knowing  this,  will  always  distribute  as  little  positive  value  as 
allowed  in  the  game,  and  retain  the  rest. 

However,  in  actual  experimental  tests  with  human  subjects,  the  veto  is  frequently  used  by 
receivers  in  response  to  low  proposals,  and  proposers  typically  offer  nearly  even  division  of 
utility — even  in  one-shot  anonymous  interactions.  Thus,  it  appears  that  people  often  prefer  less 
utility  for  themselves  in  order  to  punish  others  whom  they  perceive  as  making  unfair 
distributions;  and  it  appears  that  others  know  this  and  plan  accordingly.  One  interpretation  of  this 
behavior  is  that  individuals  in  the  role  of  the  receiver  undertake  retributive  behaviors  because 
they  are  angry.  It  is  a  likely  hypothesis  that  such  behavior  may  be  rational  in  the  long-term  if  it 
tends  to  enforce  more  equitable  distribution  for  either  that  individual  or  his  or  her  kind  or 
offspring.  Similar  kinds  of  seemingly  irrational,  but  rational  behaviors  may  arise  from  fear. 

The  general  notion  that  emotions  have  a  special  role  to  play  in  strategic  action  has 
received  growing  confirmation  in  a  range  of  other  sciences.  Neuropsychological  studies  of 
ultimatum  game  performance  have  confirmed  the  role  of  brain  substrates  of  emotion  in 
ultimatum  games  and  other  kinds  of  games  (Sanfey  et  al.,  2003;  Montague  and  Bems,  2002). 
Emotional  deficits  arising  from  brain  damage  have  been  identified  that  result  in  poor  gambling 
and  strategic  choice  behavior  (Anderson  et  al.,  1999;  Bechara  et  al.,  1994).  The  neural  substrates 
of  fear  have  become  well  understood  (Panksepp,  1998,  LeDoux,  1984),  and  the  effect  of  fear  on 
attention  and  other  capabilities  required  for  decision  making  are  increasingly  being  investigated 
(Bishop  et  al.,  2004). 

However,  progress  in  psychology  and  neural  sciences  fails  to  address  two  important 
features  of  the  role  of  emotions  in  strategic  action.  First,  existing  models  typically  have  no  way 
to  account  for  a  benefit  of  emotional  behaviors  that  stretches  across  games,  or  that  influences  the 
social  context  that  in  turn  determines  the  structure  of  future  games.  For  example,  one  utility  of 
retributive  behavior  may  be  to  enforce  social  conventions  (such  as  fairness  in  various 
distributions)  that  benefit  on  average  the  individual  in  question.  Exploring  such  a  possibility 
requires  some  modeling  of  social  interactions  over  both  a  range  of  games,  but  also  in  a  context 
where  behavior  shapes  future  interactions.  Second,  there  is  insufficient  attention  to  the  complex 
social  interactions  that  such  emotions  may  require.  We  may  need  to  explain  the  role  and  utility  ot 
a  basic  emotion  in  an  agent’s  action  not  only  in  terms  of  how  it  may  have  utility  to  the  agent 
either  immediately  or  by  helping  to  maintain  certain  social  conventions  from  which  the  agent 
benefits,  but  also  in  terms  of  how  the  display  of  the  emotion  affects  other  agents. 

Experimental  conditions  and  existing  models  are  not  yet  able  to  address  such 
possibilities.  Furthermore,  we  cannot  expect  psychology  or  neural  sciences  to  explore  these 
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problems  in  the  near  term  with  animal-subject  experiments  or  human-subject  experiments, 
because  the  requirements  of  appropriate  experiments  are  prohibitive.  It  is  very  difficult  to  induce 
fear,  anger,  and  other  emotions  reliably  and  with  controlled  and  significant  intensities  in  subjects. 
It  is  even  more  difficult  to  induce  such  emotions  in  only  selected  subjects  in  groups.  Finally,  it  is 
cost-prohibitive  to  generate  sufficient  test  cases  to  control  for  the  large  error  one  can  expect  in 
subject  interactions  because  of  their  inherit  complexity.  For  these  and  other  reasons,  the  strategic 
role  of  basic  emotions  will  remain  intractable  to  human  and  animal  experiments  for  some  time  to 
come. 

Research  Approach 

The  project  uses  a  technique  called  genetic  algorithms  to  develop  and  test  possible 
strategies  that  simple  agents  may  use  in  a  range  of  games  or  simple  strategic  situations.  This 
technique  allows  for  the  evolution  of  solutions  to  very  complex  problems.  In  this  technique, 
strategies  compete  with  each  other,  and  how  well  a  strategy  does  in  comparison  to  other 
strategies  determines  how  likely  that  strategy  is  to  have  “offspring" — that  is,  be  the  model  for 
future  identical  or  slightly  altered  strategies. 

The  problems  answered  in  this  case  were  concerned  with  when  it  is  beneficial  to  adopt 
flight  or  retributive  behavior  even  when  such  behaviors  appear  inconsistent  with  our  standard 
notions  of  rationality.  A  series  of  hypotheses  about  the  conditions  for  the  evolution  of  such 
behavior  were  tested  against  various  games  (strategic  situations)  and  various  capabilities  for  the 
agents. 


We  also  introduced  a  new  explanatory  model  into  evolutionary  game  theoretic 
explanation.  We  considered  the  possibility  that  agents  might  confront  different  types  of  games 
that  include  similar  kinds  of  interactions,  and  that  an  agent  might  develop  a  strategy  for  a  game 
that  has  highest  overall  benefit  and  then  apply  this  strategy  to  other  similar  games.  This  form  of 
explanation  provides  an  especially  compelling  form  of  explanation  for  human  performance  in  the 
Ultimatum  Game. 

Accomplishments 

Our  simulations  confirmed  our  hypotheses  that  retributive  behaviors  that  lack  short  term 
benefits  can  evolve  (win  out  as  strategies  against  other  alternative  strategies)  in  a  population  of 
interacting  agents. 

Our  primary  model  is  a  reputation-building  model  of  altruistic  retribution,  which  we  call 
the  "Reputation  Game."  In  this  model,  simple  agents  interact  with  each  other  playing  games  of 
distribution.  Each  agent  has  just  five  parameters.  Four  of  these  parameters  are  coded  as  five-bit 
binary  strings,  interpreted  as  representing  values  of  1-32  with  standard  binary  interpretation  (and 
the  addition  of  1 ).  Each  of  these  four  parameters  freely  evolved  (that  is,  the  first  generation  had 
random  values  for  each  bit  value  for  each  individual,  and  then  each  generation  is  allowed  to 
compete  to  have  “offspring,"  using  standard  genetic  algorithm  approaches  of  sexual  reproduction 
with  mutation).  These  parameters  are:  primary  proposal,  rejection-count  threshold,  secondary 
proposal  (these  three  parameters  apply  when  the  agent  is  a  proposer),  and  minimal  acceptable 


58 


proposal  (the  sole  relevant  evolving  parameter  when  this  agent  is  a  responder).  The  fifth 
parameter  is  the  rejection  count,  which  is  a  simple  integer:  this  value  begins  as  zero,  and  during  a 
mock  game  play  phase  of  thirty-two  games  as  a  responder,  this  value  is  incremented  each  time 
an  agent  rejects  an  offer.  This  can  be  thought  of  as  the  agent’s  reputation,  announced  at  the 
beginning  of  a  round,  when  the  agent  is  playing  the  role  of  responder.  During  the  actual  game 
play  phase,  games  are  played  by  each  agent  as  a  proposer,  against  some  randomly  selected  other 
agents  in  the  population.  In  each  game,  the  proposer  can  see  the  rejection  count  of  the  responder. 
If  the  responder's  rejection  count  is  greater  than  the  proposer’s  rejection-count  threshold,  the 
proposer  will  use  his  or  her  secondary  proposal.  Otherwise,  the  proposer  uses  his  or  her  primary 
proposal.  This  proposal  is  then  communicated  to  the  responding  agent.  When  playing  as  a 
responder,  the  agent  will  reject  any  offer  not  greater  than  or  equal  to  that  agent's  minimal 
acceptable  proposal  value. 

This  simulation  evolves  a  stable  strategy  of  both  offering  and  expecting  more  equitable 
distributions  in  the  Reputation  Game.  Figure  1  shows  the  mean  population  primary  and 
secondary  proposal  values  for  5,000  generations,  normalized  to  range  from  0  to  1 .  The  typical 
population,  after  300  generations,  has  both  mean  primary  and  secondary  proposal  values  of  28% 
of  utility.  The  population  mean  minimal  accept  value  settles  near  21%  of  utility. 

The  simulation  confirms  that  a  reputation  for  punishment  leads  to  a  more  equitable 
distribution  of  utility  in  games  played  with  those  who  have  a  reputation  for  retribution,  but  it  also 
shows  that  primary  proposals  (those  made  to  agents  without  a  reputation  for  retribution)  also 
evolve  in  the  population  towards  more  equitable  distributions  in  line  with  human  performance.  A 
simplistic  expectation  for  the  simulation  might  be  that  agents  will  evolve  the  lowest  possible 
primary  proposals,  but  higher  secondary  proposals.  This  way,  they  could  make  equitable 
proposals  to  retributors  but  low  proposals  to  those  without  a  reputation  for  retribution.  Instead, 
the  population  means  for  both  the  primary  and  secondary  proposals  quickly  evolve  towards  a 
more  equitable  value.  As  agents  begin  to  evolve  high  secondary  proposals,  it  becomes  in  the 
interest  of  agents  to  develop  a  reputation  over  time  for  rejecting  low  proposals.  This  creates  the 
opportunity  for  there  to  be  offspring  with  higher  minimal  accept  values.  This  also  in  turn  creates 
a  general  environment  where  offspring  are  facing  fewer  low  proposals;  so  many  agents  with 
higher  minimal  accept  values  will  not  reject  many  or  any  proposals  and  will  not  then  develop  a 
reputation  for  retribution  (their  rejection-count  will  be  low  or  zero).  The  result  is  that  reputation 
building  helps  not  only  agents  that  reject  frequently  but  also  benefits  the  entire  population.  The 
reputation  for  rejecting  low  proposals,  in  other  words,  starts  to  cast  a  halo  over  the  whole 
population,  as  even  those  with  very  few  or  no  rejections  come  to  have  higher  accept  values  and 
other  agents  make  higher  primary  proposals  because  their  ancestors  found  it  beneficial  to  avoid 
rejections.  Agents  have  an  “interest"  in  testing  the  rejection  threshold  of  other  agents,  and  this 
explains  the  wide  standard  deviation. 
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Figure  1.  Population  mean  values  for  Reputation  Game. 

Our  hypothesis  is  that  an  Ultimatum  Game  could  be  seen  as  a  kind  of  approximate 
instance  of  this  Reputation  Game.  The  Reputation  Game  is  one  shot  but  not  anonymous;  an 
anonymous  interaction  is  an  interaction  where  the  proposer  knows  nothing  of  the  responder's 
rejection  history.  If  we  interpret  this  as  the  anonymous  responder  having,  from  the  perspective  of 
the  proposer,  a  rejection  history  of  zero  (or,  alternatively,  any  rejection  history),  then  the 
proposer  will  be  offering  a  mean  value  around  9  out  of  32.  This  is  28%  of  the  utility.  Camerer's 
summary  of  Ultimatum  Game  findings  shows  a  mean  proposal  typically  ranging  from  30%  to 
40%  (Camerer,  2003).  Furthermore,  the  population  mean  minimal  acceptable  proposal  value 
tends  towards  21%  of  the  utility.  Camerer's  summary  found  that  offers  below  20%  were  rejected 
about  half  the  time,  whereas  offers  closer  to  40%  were  rarely  rejected.  Thus,  if  we  interpret 
Ultimatum  Game  performance  as  an  application  of  the  dominant  Reputation  Game  strategy  to 
the  Ultimatum  Game,  these  findings  are  close  to  human  performance. 

This  model  has  on  its  side  both  plausibility  and  parsimony.  It  is  plausible  that  human 
beings  and  their  ancestors  often  interacted  with  the  same  individuals  in  non-anonymous 
situations,  and  that  reputation  building  can  play  a  role  in  the  social  benefits  of  retribution,  so  that 
something  like  the  Reputation  Game  is  more  common  and  more  important  to  humans  than  the 
Ultimatum  Game.  The  strategy  that  agents  use  in  Ultimatum  Games  would  thus  be  a  strategy  that 
evolved  or  developed  in  response  to  something  like  Reputation  Games;  then  this  strategy  is 
applied  in  Ultimatum  Games  because  Ultimatum  Games  can  be  interpreted  as  an  instance  of 
Reputation  Games.  That  is,  there  is  a  strategy  in  the  Reputation  Game  that  can  be  interpreted  as  a 
strategy  for  an  Ultimatum  Game — in  our  case,  interpreting  the  lack  of  retribution  history  as 
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equivalent  to  a  retribution  history  of  zero  (or  any  arbitrary  value).  Thus,  the  model  can  offer  an 
explanation  of  human  Ultimatum  Game  performance  with  no  additional  hypotheses. 

Our  explanation  of  human  Ultimatum  Game  performance  thus  introduces  a  novel 
principle:  the  interpretation  of  a  game  as  an  instance  of  a  more  common  game  for  which  the 
agent  has  an  existing  strategy.  The  kind  of  evolutionary  game  theory  explanation  simulated  here 
is  neutral  regarding  whether  it  describes  a  strategy  that  is  inherited  or  culturally  transmitted,  but 
depends  upon  the  presumption  that  agents  acquire  and  retain  a  strategy  in  a  way  resistant  to 
immediate  learning.  The  ancestors  of  human  beings  lived  in  socially  rich  environments  where 
most  strategic  interactions  were  with  known  individuals  with  whom  one  was  likely  to  interact 
again  in  the  future.  It  is  for  this  reason  that  we  assume  that  the  Reputation  Game  is  much  more 
representative  of  a  typical  human  interaction  than  is  an  Ultimatum  Game.  The  model  is  neutral 
regarding  whether  the  evolution  of  the  strategies  is  realized  through  inherited  structures,  or 
cultural  transmission  of  behaviors,  or  any  other  form  of  transmission.  Rather,  all  that  is  supposed 
by  the  model  is  that  strategies  are  repeated  and  variations  of  them  attempted,  and  the  strategies 
that  are  reproduced  tend  to  be  those  that  beat  out  other  strategies.  Consistent  with  this  form  of 
explanation,  once  a  strategy  is  adopted  by  an  agent,  it  may  be  the  sole  strategy  that  the  agent  uses 
throughout  all  game  play  in  the  Reputation  Game  and  similar  games  or  approximate  instances. 
That  is,  there  is  no  reason  to  suppose  that  the  evolution  of  strategies  is  occurring  within  any 
particular  agent;  the  more  simple  application  of  the  model  is  that  each  agent  is  using  one 
strategy.  This  allows  the  model  to  explain  behavior  in  various  instances:  Immediate  learning  is 
not  a  factor  in  the  explanation. 

This  explanation  is  consistent  with  a  growing  body  of  evidence  concerning  human 
emotions.  An  important  feature  of  anger  and  other  motivations  is  that  many  of  our  emotions 
appear  revision  resistant;  that  is,  unlike  the  kinds  of  preferences  that  economists  typically  study, 
they  cannot  easily  be  altered.  For  this  reason  a  number  of  economists  have  argued  that  emotions 
can  act  as  special  commitment  mechanisms,  enforcing  behavior  that  would  otherwise  be  less 
likely  to  occur  (Frank,  1989;  Hirshleifer,  1984).  Consistent  with  this,  neural  scientists  have 
uncovered  a  growing  body  of  evidence  that  some  emotions  and  other  forms  of  preferences  are 
significantly  independent  of  our  abilities  to  consciously  process  information  (Zajonc,  1968; 
Kunst-Wilson  and  Zajonc,  1980;  Ohman,  1988;  Ohman  et  al.,  1989;  Ohman  and  Soares,  1993, 
1994).  An  inherited  pancultural  disposition  could  behave  in  this  way,  and  remain,  because  its 
resistance  to  revision  was  not  harmful  or  was  even  beneficial.  Finally,  many  species  of 
organisms  have  invested  significant  resources  into  the  ability  to  display  threats,  suggesting  that 
the  role  of  making  credible  threats  is  an  important  survival  skill.  Organisms  find  it  worthwhile  to 
expend  energy  to  create  displays  meant  to  convey  their  ferocity  and  willingness  to  fight.  Lions 
growl,  dogs  bare  their  teeth,  birds  inflate  themselves  to  exaggerate  their  size,  and  so  on.  These 
displays  may  be  interpreted  as  attempts  to  create  and  convey  a  reputation  of  being  willing  to 
retribute,  avoiding  some  conflicts  but  also  in  some  cases  perhaps  earning  more  equitable 
treatment  from  other  conspecifics.  These  findings  are  consistent  with  the  idea  that  anger  acts  as 
an  inherited  disposition  to  retribute,  that  we  benefit  from  communicating  this  disposition,  and 
that  we  may  act  on  this  disposition  even  in  cases  where  the  benefits  of  reputation  are  lost  because 
the  disposition  is  not  easily  revised  or  controlled. 
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Contributions  to  Basic  Science 


The  project  provides  a  new  model  of  altruistic  retribution  and  extends  standard  game- 
theoretic  or  decision-theoretic  approaches.  The  interpretation  of  the  primary  model  suggests 
several  new  areas  for  future  empirical  human  subject  research.  First,  the  model  of  retributive 
reputation  building  predicts  that  over  very  many  anonymous  interactions,  less  equitable 
distributions  of  goods  would  come  to  dominate.  This  could  be  tested  empirically.  Second,  the 
model  predicts  that  certain  forms  of  rational  control  of  emotion  (those  inconsistent  with  the 
primary  inherited  function  of  the  emotions)  will  be  difficult,  and  this  is  because  of  a  biologically 
inherited  resistance  to  reason  that  is  beneficial  to  the  individual.  Finally,  the  model  introduces  a 
novel  explanatory  tool  for  evolutionary  game  theory:  the  interpretation  of  a  game  as  being  an 
instance  of  a  more  common  and  general  game. 

Potential  Army/Military  Applications 

These  results  suggest  that  fear  and  anger  must  be  recognized  as  inherited  strategies  in 
order  to  be  effectively  managed.  These  basic  emotions  resist  rational  control  not  because  they  are 
some  kind  of  biological  left-over  which  has  not  become  amenable  to  our  rational  abilities — as  if 
they  were  a  kind  of  appendix  of  the  mind.  Rather,  they  resist  rational  control  because  it  is  part  of 
their  essential  benefit  to  do  so.  This  benefit  is  seen  in  at  least  two  ways. 

First,  short-term  and  seemingly  irrational  behaviors  that  signal  and  establish  a  reputation 
for  retributive  anger  can  be  an  important  way  to  make  credible  threats.  It  is  necessary,  that  is,  for 
anger  to  resist  reason  in  order  for  it  to  allow  certain  kinds  of  credible  threats.  The  same  is  likely 
true  of  fear  also.  For  example,  fear  could  serve  to  allow  an  individual  to  make  credible  threats  of 
defection  even  in  the  face  of  serious  (deferred)  penalties. 

Second,  anger  and  fear  may  have  a  natural  inherited  tendency  to  resist  rational  control  for 
the  very  reason  that  fitness  and  rationality  can  diverge.  It  might  be  irrational,  given  someone's 
beliefs,  to  avoid  a  dangerous  situation  (e.g.,  he  or  she  may  sincerely  believe  that  some  action  is 
more  important  than  the  expected  threat  to  his  or  her  life  the  action  will  cause),  but  it  may  still  be 
biologically  fit  for  that  person  to  avoid  the  action.  Thus,  for  example,  fear  may  in  part  resist 
rational  control  because  this  has  been  sufficiently  fitness  improving. 

Recognizing  these  facts  can  ensure  that  it  is  understood  that  these  emotions  are 
inevitable,  and  cannot  be  managed  through  direct  rational  argument.  Rather,  these  are  inherited 
strategic  benefits  that  must  be  considered  as  universal  elements  of  human  social  interaction 
between  groups  and  between  individuals  within  a  group. 
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Problem(s)/Research  Question(s)  -  Can  a  racially  unbiased  test  of  the  ability  to  process 
information  predict  how  well  young  adults  will  succeed  in  college  classes? 

Technical  Barrier(s)  -  Current  theories  of  intelligence  are  based  on  an  assumption  that  all  those 
taking  IQ  tests  have  had  equal  opportunity  for  exposure  to  the  information  being  tested.  Thus, 
past  efforts  to  develop  an  intelligence  test  that  is  culture-fair  have  not  been  successful. 

Significance/Impact  for  Basic  Research  -  The  present  study  provides  further  evidence  to 
evaluate  a  theory  that  defines  intelligence  as  information  processing  ability  (Fagan,  1992,  2000). 
Specifically,  how  much  you  know  (your  IQ)  depends  not  only  on  how  well  you  process 
information  (your  intelligence)  but  also  on  the  information  given  to  you  by  your  culture  to 
process.  Current  research  on  a  theory  of  intelligence  as  information  processing  finds  racial 
differences  in  IQ  to  be  due  to  cultural  factors.  A  test  of  information  processing  promises  to  be  the 
first  valid,  culture-fair  test  of  intelligence. 

Potential  Transitions  -  The  results  of  this  research  will  aid  in  gaining  the  objectives  of: 

•  The  Army’s  Science  and  Technology  Objectives  (STO  IV.  SP.  2002.02  and  DTO  HS.52 
and  WP  1 1 1)  as  to  Training  of  Future  Combat  Commanders  and  Staff; 

•  The  Army’s  Science  and  Technology  Objectives  (STO  IV.  SP.  2003.06  and  WP  215)  for 
Future  Force  Warrior  Training; 

•  The  Army’s  Science  and  Technology  Objectives  (STO  IV.  SP.  2002.01  and  DTO  HS.54 
and  WP  104  and  WP  257)  as  to  Selection,  Classification  and  Performance  Metrics  for  the 
Objective  Force;  and 

•  ARI's  Work  Programs  on  Leadership  (WP  103, 211,218,  269,  and  285). 

Correspondence  to  the  Aims  of  the  ARI  Basic  Research  Program  -  This  includes: 

1 .  The  general  purpose  of  the  present  contract  is  to  add  fundamental  knowledge  to 
behavioral  science  ...and  to  discover  general  principles.. .  [through  an]  integrated 
programmatic  effort  to  develop  and  to  test  theory  (as  called  for  by  BRU  in  BRO-BAA, 
page  1 ). 

2.  The  specific  goal  is  to  discover  the  relationships  among  information  processing  ability 
and  knowledge  that  lead  to  achievement  and  productivity  in  complex  situations.  The  goal 
is  in  keeping  with  the  Army’s  goal  to  improve  its  ability  to  Select,  classify,  train,  and/or 
develop  Soldiers  and  leaders  who. ..are  adaptable.  ..[and  who] . .  .can  function  effectively 
in. ..information  rich  ...environments  (BRO-BAA,  page  1,  II,  1,  a,  b). 
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3.  One  of  the  BRO-BAA  Basic  Research  Areas  of  Interest  is  leadership  skills.  The  proposed 
research  will  aid  in  the  goal  of  identifying  Leadership  skills  in  Adaptability  ...when  faced 
with  novel  situations  (BRO-BAA,  page  4,  III,  C,  1 ). 

4.  The  research  has  resulted  in  a  reliable,  valid,  and  culture-fair  test  of  intelligence  based  on 
the  ability  to  process  new  information.  The  research  aids  in  further  demonstrating  that 
intelligence  is  a  function  of  an  aptitude  [learning  ability  in  novel  situations]  that  cuts 
across  domains  and  how  to  develop  a  method  for  measuring  this  aptitude  [and  to  further] 
validate  [this]  cognitive  ability  against  relevant  criteria ,  i.e.,  academic  achievement 
(BRO-BAA.  page  5,  III,  D,  1). 

In  summary,  the  research  is  in  keeping  with  the  Army's  goal  to  select  Soldiers  and 
leaders  who  can  function  effectively  in  information  rich  environments  (BRO-BAA,  page  1,  II,  1, 
a,  b)  and  to  identify  leadership  skills  as  to  adaptability  when  faced  with  novel  situations  (BRO- 
BAA,  page  4,  III,  C,  1).  The  proposed  research  will  further  aid  in  demonstrating  that  intelligence 
defined  as  learning  ability  in  novel  situations  is  an  aptitude  that  crosses  domains,  and  in 
validating  this  cognitive  ability  against  relevant  criteria  (BRO-BAA,  page  5,  III,  D,  1 ). 

Overview 

The  present  study  is  based  on  a  theory  (Fagan,  1992,  2000)  that  assumes  that  we  act  on 
the  basis  of  what  we  know  and  that  what  we  know  is  a  result  of  our  processing  of  the  information 
provided  to  us  by  our  culture.  Based  on  these  assumptions,  the  IQ  score  is  seen  as  a  measure  of 
knowledge.  Specifically,  how  much  you  know  (your  IQ)  depends  on  how  well  you  process 
information  (your  intelligence)  and  also  on  the  information  given  to  you  by  your  culture  to 
process.  The  theory  has  been  used  to  explain  the  source  of  IQ  differences  between  Americans  of 
different  races.  Specifically,  we  have  found  (Fagan  &  Holland,  2002,  2003,  2004)  that  American 
Whites  and  African-Americans  who  differ  in  IQ  do  not  differ  in  their  ability  to  process 
information  when  that  information  has  been  equally  accessible  to  both  groups. 

Research  Approach 

College  students  are  given  multiple  choice  tests  of  their  ability  to  acquire  new 
information  concerning  the  meanings  of  previously  unknown  words,  sayings,  similarities,  and 
analogies.  They  are  also  tested  for  their  knowledge  of  vocabulary  in  general,  opposites,  and 
analogies.  An  entire  session  (training  and  testing)  lasts  about  30  minutes.  With  the  students' 
written  consent,  Scholastic  Assessment  Test  scores.  Grade  Point  Averages  (GPAs)  and 
numerical  grades  in  a  course  also  are  obtained.  Associations  between  performance  on  the 
culture-fair  tests,  academic  aptitude  (Scholastic  Assessment  Test  scores),  and  specific 
achievement  (cumulative  GPAs  and  objective  test  scores  in  college  courses)  are  analyzed. 

Accomplishments 

The  contract  was  undertaken  to  discover  if  a  valid,  culture-fair  test  of  intelligence  based 
on  the  ability  to  process  new  information  could  be  developed.  Our  initial  results,  reported  to  ARI 
on  8/18/05,  were  based  on  a  sample  of  421  students,  drawn  from  a  private  university,  two 
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colleges  and  a  two-year  community  college,  who  participated  during  the  initial  1 0-month  period 
of  the  contract.  We  found  that  the  tests  of  new  learning,  developed  to  that  point,  were  culture- 
fair,  reliable,  and  predictive  of  both  academic  achievement  (numerical  grades  in  class)  and  of  a 
brief  version  of  a  standard  test  of  scholastic  aptitude  (the  Scholastic  Assessment  Test  -Verbal 
[SAT-V]). 

Continued  testing  since  8/05,  using  a  revised  version  of  the  test  of  new  learning,  has 
involved  a  higher  percentage  of  community  college  students  (relative  to  those  in  the  private 
university  and  colleges)  than  was  possible  during  the  initial  contract  period.  A  wider  range  of 
new  learning,  performance  on  the  brief  SAT-V,  and  academic  achievement  scores  have  allowed 
a  more  accurate  determination  of  the  role  of  information  processing  (new  learning)  and  the  role 
of  the  existing  knowledge  base  (e.g.  brief  SAT-V  scores)  in  the  determination  of  academic 
achievement  (grades).  The  results  of  the  continued  testing  are  now  based  on  completed  protocols 
from  596  students  which,  together  with  the  sample  of  421  students  already  noted,  yields  a  final 
total  of  1,017  participants.  (The  planned  sample  size  was  900  participants).  The  latest  results  tell 
us,  in  accord  with  our  previous  studies,  that  tests  of  new  learning  are  culture-fair.  African- 
Americans  perform  as  well  as  whites.  Further  they  demonstrate  again  that  tests  of  new  learning 
and  tests  of  existing  knowledge  each  contribute  independent  variance  to  the  prediction  of  class 
grades,  R  (2/593)  =  .47,  P<.0001 ,  with  Betas  of  .12  (t  =  2.4,  P  <  .017)  and  .39  (/  =  8.3,  P  < 
.0001),  for  predictions  from  new  learning  and  existing  knowledge,  respectively. 

Two  other  findings  are  worthy  of  note.  The  first  is  that  a  subset  of  1 8  of  the  30  items 
tested  on  the  tests  of  new  learning  proved  to  be  particularly  predictive  of  achievement,  r  (432)  = 
.47,  P  <  .0001,  and  were  solved  equally  well  by  members  of  different  racial  groups.  These  1 8 
items  may  constitute  a  test  of  new  learning  to  be  employed  in  future  studies  where  the  inclusion 
of  other  measures  in  a  protocol  may  be  done  while  keeping  session  length  within  the  temporal 
bounds  comfortable  for  participants. 

The  second  notable  finding  is  that  a  brief,  24-item  version  of  the  SAT-V  that  we  created 
turns  out  to  be  as  predictive  of  college  grades  as  the  standard  SAT-V.  We  constructed  the  brief 
SAT-V  because  our  community  college  participants  are  not  required  to  take  either  the  SAT  or 
the  ACT  for  admission  to  college  and  we  wanted  a  measure  of  knowledge  that  would  be 
common  to  both  community  college  students  and  university  students.  We  also  wanted  a  measure 
that  would  be  comparable  to  the  standard  SAT-V  as  to  validity.  A  sub-sample  of  371  students 
attending  the  private  universities  provided  the  data  for  a  comparison  of  our  brief  SAT-V  and  the 
Standard  SAT-V  as  to  their  comparative  validity  in  predicting  academic  achievement.  Measures 
of  specific  academic  achievement  (exam  performance  on  objective  tests)  were  obtained  (with  the 
students'  permission)  from  the  instructors  of  their  courses.  Cumulative  Grade  Point  Averages 
representing  at  least  two  semesters  of  attendance  were  also  obtained  (with  the  students' 
permission)  from  the  registrar  at  each  institution. 

Both  the  standard  SAT-V  and  the  brief  SAT-V  test  were  successful  in  predicting 
academic  performance.  Of  most  importance  is  the  fact  that  the  predictions  from  each  test  to 
academic  performance  were  virtually  identical.  No  significant  difference  was  found  between  the 
SAT-V  prediction  of  r  =  .30  to  class  grade  or  the  brief  SAT-V  test’s  prediction  of  r  =  .28  to  class 
grade,  t  (364)  =  0.6;  similarly  so  for  the  predictions  of  each  measure  of  aptitude  to  cumulative 
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GPA  (r  =  .29  and  r  =  .26,  respectively),  with  1  (366)  =  0.9.  The  results  tell  us  that  the  scores 
derived  from  our  brief  version  of  the  type  of  questions  typically  asked  on  the  SAT-V  are  as 
predictive  of  academic  performance  in  college  as  are  scores  derived  from  the  SAT-V  itself. 

Currently,  questions  have  been  raised  in  a  series  of  articles  in  the  New  York  Times  (e.g., 
Lewin,  2006)  about  the  amount  of  time  necessary  to  complete  the  SAT,  which  is  estimated  at  3 
hours  and  45  minutes  (and  longer  for  students  with  disabilities).  Earlier  reports  in  the  New  York 
Times  (e.g.,  Arenson,  2005)  revealed  a  sizable  error  rate  in  the  current  scoring  process  for  the 
SAT.  In  the  present  ARl-sponsored  study,  we  found  that  a  very  brief,  easily  scored  version  of  the 
SAT-V  would  be  as  predictive  of  college  grades  as  the  standard  SAT-V. 

In  summary,  we  find  that  there  are  at  least  two  factors  which  determine  success  in  a 
complex  learning  environment  such  as  higher  education:  One  is  information  processing  (new 
learning)  ability  and  the  other  is  the  extent  of  knowledge  one  has  acquired  to  that  point. 

Contributions  to  Basic  Science 

Jensen  (1998)  believes  that  the  IQ  score  reflects  what  he  calls  the  general  factor  (or  #) 
underlying  intelligence.  Differences  in  IQ  scores  are  due  to  differences  in  g.  According  to 
Jensen,  genetic  factors  play  a  primary  role  in  determining  g.  Thus,  Jensen  (2000)  believes  that  it 
may  not  be  possible  to  come  up  with  tasks  that  show  no  differences  in  test  performance  between 
Blacks  and  Whites  and  yet  still  predict  academic  performance.  Fagan  (2000)  does  not  agree  with 
Jensen,  pointing  out  that  the  failure  to  develop  tests  of  intelligence  that  can  be  fairly  applied 
across  racial  groups  stems  from  a  theoretical  bias  to  equate  the  IQ  score  with  intelligence  rather 
than  with  knowledge.  If  intelligence  is  defined  as  information  processing  and  the  IQ  score  as 
knowledge,  the  possibility  of  culture-fair  tests  of  intelligence  based  on  estimates  of  information 
processing  arises.  Fagan  is  not  alone  in  such  a  conjecture.  Williams  (2000,  p.  1 7)  notes  that  “ 
Fagan's  ideas”  (Fagan,  2000)  of  measuring  thinking  or  information  processing  rather  than 
accumulated  knowledge  are  relevant  to  the  debate  on  intelligence  testing  and  affirmative  action 
because.. .a  true  measure  of  processing  efficiency  (if  it  could  be  devised)  would  be  fair  to 
members  of  all  racial  and  ethnic  groups.”  The  chief  theoretical  implication  of  the  results  of  the 
present  study  is  that  the  assumption  of  equal  opportunity  for  exposure  to  information  made  by 
theories  of  intelligence  is  false.  When  such  an  assumption  is  shown  to  be  false,  it  becomes 
possible  to  develop  culture-fair  tests  of  intelligence  that  allow  basic  intellectual  abilities  to  be 
recognized.  Specifically,  the  inequality  of  educational  achievement  among  races  in  our  country 
has  highlighted  a  need  for  culture-fair  tests  of  intelligence.  For  example.  Blacks  do  not  do  as  well 
as  Whites  on  IQ  tests  and  other  tests  of  knowledge  such  as  the  Scholastic  Assessment  Test. 

Basing  admission  to  higher  education  on  such  test  scores  means  that  only  a  small  percentage  of 
Blacks  are  eligible  for  admission  to  colleges  and  universities.  A  culture-fair  test  of  intelligence 
would  allow  basic  abilities  to  be  measured  and  would  allow  those  with  appropriate  intellectual 
skills  to  pursue  further  schooling. 

Potential  Army/Military  Applications 

Two  of  the  guiding  missions  of  ARI  both  historically  and  currently  are  to  provide  new 
technology  to  meet  the  personnel  challenges  of  the  Army  in  recruiting,  selecting,  and  assigning 
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quality  Soldiers  and  to  provide  scientific  insight  into  leader  development.  The  present  research 
provides  basic  scientific  data  that  may  be  applied  by  ARI  to  aid  both  in  Soldier  selection . 
assignment,  and  performance  and  in  leader  development.  The  results  of  the  present  contract  may 
lead  to  applied  research  that  would  be  meaningful  to  the  Army  in  the  development  of  new, 
applied  behavioral  technologies  in  order  to  identify  individual  differences  in  cognitive  ability 
that  may  be  used  in  the  selection  of  Soldiers  and  in  the  identification  of  mature,  effective  leaders. 
In  addition,  the  proposed  research  will  provide  data  to  aid  such  selection  and  identification  to  be 
culture-fair  with  regard  to  sex  and  race.  In  general,  tests  of  information  processing  ability  that  are 
valid  predictors  of  achievement  in  complex  situations  will  have  important  military  application. 
Such  tests  can  aid  in  providing  a  valid  means  to  evaluate  volunteers  and  select  candidates  for 
advanced  education  or  training  in  complex  situations  of  the  sort  performed  by  Army  personnel. 
Such  tests  can  also  be  employed  in  further  selection  and  training  stages.  In  the  field,  such  skills 
are  also  necessary  to  make  quick  and  correct  decisions  based  on  rapidly  incoming  information. 
Skill  in  information  processing  is  likely  to  be  a  key  indicator  of  effective  future  Army  leaders. 
Finally,  using  tests  of  information  processing  known  to  be  culture-fair  may  provide  an  increase 
in  the  number  of  eligible  recruits  and  an  incentive  to  re-enlistment  on  the  part  of  minorities. 

Specific  transitions  to  applied  research  include: 

1 .  Selection,  Classification,  &  Performance  Metrics  for  the  Future  Force  Soldier  (STO  IV. 
SP.  2002.1 ).  Findings  from  our  current  research  on  the  role  of  information  processing 
ability  can  be  applied  to  a  personnel  selection  and  classification  system  geared  [toward 
identifying  Soldiers  that  are]  self-directed  and  effective  information  processors  and 
problem  solvers,  [a  system  that]  might  be  incorporated  into  future  enlisted  selection  and 
classification  procedures  [as  well  as  being  used  as]  outcome  measures  for  future  Soldier 
training  and  assessment. 

2.  Performance  Measures  for  2 1  st  Century  Soldier  Assessment.  Our  findings  may  be 
applied  toward  the  development  of  a  cost-effective  ...assessment  system. ..for  certifying  a 
Soldier 's  qualification  for  job  performance  [and  aid  in  the  provision  of] 
recommendations  for  an  Army-wide  system  to  certify  the. ..qualifications  of  Soldiers  for 
job  performance. 

3.  Future-Train:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat  Team 
Commanders  and  Staffs.  The  research  focuses  on  how  to  measure  cognitive  skills  that 
interact  to  influence  performance  in  complex  situations.  Such  aptitude  may  underlie  the 
skills  of  future  commanders  and  staff. 

4.  Leader  Development,  Accelerating  Leader  Development.  The  present  research  improves 
the  assessment  of  cognitive  skills,  which  will  prove  to  be  of  importance  to  the  selection 
and  development  of  leaders. 

5.  Data  on  predictive  validity  relative  to  test  length  may  be  of  benefit  to  the  ARI  applied 
program  for  Selection,  Classification,  &  Performance  Metrics  for  the  Future  Force 
Soldier  (STO  IV.  SP.  2002.1). 
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Testing  a  Theory  of  the  Determinants  of  Individual  Job  Performance 
for  United  States  Army  Junior  Commissioned  Officers 


Contract  #:  W74V8H-05-K-0005 
Institution:  Personnel  Decisions  Research 
Institutes  (PDRI) 


Contract  Dates:  01/10/2005  to  09/30/2007 

PI:  Jeff  W.  Johnson 

Liaison:  Robert  Kilcullen,  SARU 


Problem(s)/Research  Question(s)  -  How  do  individual  difference  variables  work  to  influence 
performance  on  specific  performance  dimensions  relevant  to  junior  commissioned  officers  in  the 
U.S.  Army? 

Technical  Barrier(s)  -  There  is  no  well-established  theory  of  the  process  by  which  individual- 
difference  variables  predict  citizenship  performance  or  adaptive  performance,  in  either  a  military 
or  a  civilian  context. 

Significance/Impact  for  Basic  Research  -  Support  for  this  theory  of  performance  would 
contribute  to  a  better  understanding  of  the  relationships  between  predictors,  mediators,  and  job 
performance  criteria. 

Potential  Transitions  -  ARI  has  created  various  temperament  scales  thought  to  be  related  to 
adaptive  performance  and  has  empirically  linked  those  measures  to  the  mission  performance  of 
Special  Forces  (SF)  Soldiers.  ARI  is  now  leveraging  this  measure  to  create  a  self-development 
feedback  packet  for  SF  Officers.  The  measures  created  in  this  basic  research  project  are  designed 
to  predict  performance  at  a  more  proximal  level  than  the  current  temperament  scales,  so  they 
could  be  used  to  revise  or  improve  this  self-development  package  or  other  Soldier  self¬ 
development  packages  that  are  used  outside  of  SF. 


Overview 


Organizational  citizenship  performance  and  adaptive  performance  are  increasingly 
important  components  of  the  Army  junior  commissioned  officer  job  performance  construct 
domain.  Despite  the  importance  of  these  constructs,  there  has  been  very  little  research  studying 
them  in  a  military  setting.  There  is  also  no  well-established  theory  of  the  process  by  which 
individual-difference  variables  predict  citizenship  performance  or  adaptive  performance,  in 
either  a  military  or  a  civilian  context.  In  this  research,  we  are  testing  a  theory  of  the  process 
through  which  individual  difference  variables  work  to  influence  performance  on  specific 
citizenship  and  adaptive  performance  dimensions  relevant  to  junior  commissioned  officers  in  the 
U.  S.  Army.  Johnson  (2003)  proposed  a  theory  of  performance  determinants  that  goes  beyond 
earlier  theories  proposed  by  Campbell,  McCloy,  Oppler,  and  Sager  (1993)  and  Motowidlo, 
Borman,  and  Schmit  (1997),  primarily  by  specifying  an  expanded  conceptualization  of 
motivation.  Johnson  broke  the  motivation  component  into  specific  motives  (e.g.,  organizational 
commitment,  job  satisfaction),  proactive  cognitive  processes  (e.g.,  self-efficacy,  goal 
commitment),  and  on-line  cognitive  processes  (e.g.,  self-regulation).  These  variables,  along  with 
knowledge,  skill,  and  habits,  mediate  the  relationship  between  indirect  performance  determinants 
(e.g.,  cognitive  ability,  personality)  and  any  kind  of  individual  job  performance  construct. 
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Support  for  this  theory  would  provide  researchers  and  practitioners  with  a  tool  that  can  be 
used  to  identify  the  constructs  through  which  individual  difference  variables  work  to  influence 
performance  on  specific  dimensions.  This  is  especially  important  for  identifying  predictors  of 
citizenship  and  adaptive  performance.  Such  a  tool  would  contribute  to  a  better  understanding  of 
the  relationships  between  predictors,  mediators,  and  job  performance  criteria,  and  could  be  used 
to  choose  appropriate  predictors  for  whatever  criterion  construct  is  of  interest  for  a  particular 
Army  job.  A  rigorously  developed  and  tested  theory  of  the  influence  of  individual  differences  on 
job  performance  would  not  only  facilitate  selection,  classification,  and  training  of  junior 
commissioned  Army  officers,  but  would  also  add  significantly  to  the  evolving  understanding  of 
job  performance. 

Research  Approach 

In  order  to  test  this  theory,  we  assembled  and  developed  a  battery  of  instruments  that  are 
construct- valid  measures  of  each  component  of  the  theory.  All  measures  created  for  this  project 
are  based  on  input  from  ROTC  cadets.  To  measure  task,  citizenship,  and  adaptive  performance, 
we  created  a  multisource  performance  rating  instrument  measuring  performance  on  five  different 
dimensions.  We  created  a  past  behavior  record  to  measure  skill  and  a  situational  judgment  test  to 
measure  knowledge,  both  of  which  are  relevant  to  five  performance  dimensions.  We  also  created 
(a)  a  unique  self-report  measure  of  work  habits;  (b)  a  motives  scale  assessing  attitudes,  values, 
and  preferences  that  predict  performance;  and  (c)  a  motivation  scale  measuring  self-efficacy, 
expectancy,  goal  content,  and  goal  commitment.  We  also  created  a  self-regulation  scale  to 
measure  the  extent  to  which  respondents  were  able  to  focus  on  performing  the  behaviors  they 
indicated  they  intended  to  perform.  We  also  assembled  a  battery  of  existing  instruments  to 
measure  cognitive  ability,  personality,  experience,  and  action  control. 

The  predictor  battery  was  administered  to  155  ROTC  cadets  in  the  Fall  of  2006.  Criterion 
administration  took  place  during  March  and  April  of  2007,  allowing  us  to  assess  the  extent  to 
which  cadets  performed  the  behaviors  they  intended  to  perform.  The  self-regulation  scale  was 
administered  four  times  during  this  period,  allowing  us  to  track  how  well  each  cadet  was  able  to 
maintain  focus  on  performing  these  behaviors.  Each  cadet  was  rated  by  approximately  four 
peers.  These  data  will  be  analyzed  using  structural  equation  modeling  to  test  the  Johnson  (2003) 
theory,  as  well  as  several  ancillary  hypotheses. 

Accomplishments 

Accomplishments  to  date  have  been  (a)  the  determination  of  the  performance  dimensions 
to  be  studied  in  this  project,  (b)  the  development  of  the  instruments  designed  to  measure  each 
component  of  the  theory,  (c)  the  recruitment  of  cadets  and  midshipmen  in  ROTC  programs  to 
participate  in  the  research,  and  (d)  data  collection. 

The  performance  dimensions  were  identified  by  presenting  a  list  of  specific  performance 
dimensions  relevant  to  task,  citizenship,  and  adaptive  performance  to  a  group  of  nine  ROTC 
cadets.  Through  group  discussion,  we  identified  one  task  performance  dimension,  three 
citizenship  performance  dimensions,  and  one  adaptive  performance  dimension  that  (a)  are 
especially  relevant  to  ROTC  cadet  performance,  (b)  are  observable  by  other  cadets,  and  (c) 
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display  variation  in  the  level  of  performance  among  cadets.  The  performance  dimensions  are  (a) 
decision  making/problem  solving,  (b)  maintaining  good  working  relationships,  (c)  organizational 
commitment,  (d)  showing  initiative,  and  (e)  adapting  to  changing  or  uncertain  situations. 

We  developed  the  following  instruments: 

•  Performance  measurement  instrument  -  This  instrument  consists  of  37  items  measuring 
cadet  performance  on  the  five  performance  dimensions.  Each  item  is  a  behavioral 
statement  that  represents  effective  performance  on  one  of  the  performance  dimensions. 
Raters  will  use  a  5-point  rating  scale  to  rate  the  extent  to  which  the  statement  describes 
the  individual  being  rated. 

•  Experience  Inventory  -  This  instrument  consists  of  the  same  items  that  are  on  the 
performance  measurement  instrument,  and  respondents  indicate  the  degree  of  experience 
they  have  performing  each  behavior. 

•  Past  Behavior  Record  -  This  instrument  measures  skill  relevant  to  the  five  performance 
dimensions.  The  individual  describes  a  situation  he  or  she  has  experienced  that  elicited 
behavior  relevant  to  each  of  the  performance  dimensions  and  the  behavior  he  or  she 
exhibited.  Trained  raters  used  standardized  rating  scales  to  evaluate  the  level  of  skill  that 
each  behavior  represents.  Two  types  of  rating  scales  were  developed  to  determine  which 
yields  scores  that  better  predict  performance. 

•  Situational  Judgment  Test  -  This  instrument  measures  knowledge  relevant  to  the 
performance  dimensions.  Each  item  consists  of  a  situation  and  a  set  of  response  options 
that  vary  in  their  effectiveness.  The  respondent  uses  a  7-point  scale  to  rate  the  level  of 
effectiveness  each  response  represents,  as  well  as  choosing  the  most-  to  least-effective 
response. 

•  Military  Motives  Scale  -  This  instrument  measures  values,  interests,  preferences,  and 
attitudes  that  are  expected  to  predict  behavior  relevant  to  the  five  performance 
dimensions.  This  instrument  includes  measures  of  job  satisfaction;  military  values; 
affective  and  continuance  commitment;  the  citizenship  motives  of  organizational 
concern,  prosocial  values,  and  impression  management;  and  interest  in  being  in  situations 
that  require  behavior  relevant  to  the  performance  dimensions. 

•  Work  Habits  Scale  -  This  instrument  measures  the  extent  to  which  behaviors  relevant  to 
each  of  the  performance  dimensions  are  consistent  with  the  respondent's  habitual  way  of 
doing  things. 

•  Proactive  Cognitions  Scale  -  This  instrument  includes  questions  about  expectations,  self- 
efficacy,  and  intentions  to  perform  behaviors  relevant  to  the  performance  dimensions. 

•  Self-Regulation  Inventory  -  This  instrument  measures  the  extent  to  which  participants 
were  able  to  maintain  their  intentions  to  behave  in  a  certain  way  when  presented  with  the 
opportunity  over  the  previous  four  weeks. 


70 


We  recruited  participants  from  Army,  Navy,  and  Air  Force  ROTC  programs  at  six 
different  universities.  We  also  obtained  IRB  approval  to  conduct  the  research  at  each  university. 
The  universities  were: 

•  University  of  Minnesota  (Army,  Navy,  Air  Force) 

•  University  of  South  Florida  (Army) 

•  University  of  North  Carolina-Charlotte  (Air  Force) 

•  Colorado  State  University  (Army,  Air  Force) 

•  Michigan  State  University  (Air  Force) 

•  Purdue  University  (Army,  Air  Force) 

We  administered  one  set  of  predictor  instruments  in  in-person  testing  sessions,  and  the 
remainder  were  administered  online.  We  obtained  complete  predictor  data  from  1 55  respondents. 
Those  respondents  also  completed  four  Self-Regulation  Inventories  approximately  once  a  month 
following  predictor  administration.  Criterion  administration  began  in  mid-March  and  is  ongoing. 

Contributions  to  Basic  Science 

This  project  will  advance  our  understanding  of  job  performance  prediction  by  providing 
evidence  in  support  of  a  theory  of  the  process  by  which  individual-difference  variables  predict 
specific  dimensions  of  performance.  Many  theories  of  the  relationship  between  individual 
differences  and  job  performance  have  been  proposed,  but  there  is  very  little  empirical  support  for 
any  of  them.  This  is  especially  true  for  less-studied  aspects  of  performance  such  as  citizenship 
performance  and  adaptive  performance,  which  are  the  primary  focus  of  this  research.  An 
established  model  of  the  process  by  which  individual  differences  in  predictor  variables  lead  to 
individual  differences  in  specific  dimensions  of  performance  can  be  used  to  guide  research 
linking  specific  predictors  to  specific  performance  dimensions  by  helping  to  identify 
theoretically  relevant  predictors  for  different  criteria.  In  the  proposed  model,  the  construct  of 
motivation  is  expanded  to  highlight  how  different  predictors  influence  different  components. 
Self-regulation  is  the  primary  component  that  previous  models  were  missing.  This  construct  is 
very  important  because  it  (a)  is  strongly  related  to  personality;  (b)  helps  explain  how  people  with 
similar  knowledge,  ability,  goals,  and  desire  to  perform  differ  in  their  level  of  performance;  and 
(c)  helps  explain  how  people  overcome  conflicting  habits  to  perform  in  accordance  with  their 
goals. 


This  project  also  will  yield  several  potentially  useful  products.  We  created  a  multisource 
performance  rating  instrument  to  measure  task,  citizenship,  and  adaptive  performance.  These 
performance  components  are  important  aspects  of  the  Army  junior  commissioned  officer 
performance  construct  domain.  We  created  a  situational  judgment  test  (SJT)  to  assess  knowledge 
relevant  to  the  performance  dimensions  chosen  for  study.  The  SJT  presents  the  individual  with  a 
situation  and  a  set  of  response  options  varying  in  their  effectiveness.  We  developed  a  past 
behavior  record  to  measure  skill  relevant  to  the  targeted  performance  dimensions.  Using  a  highly 
structured  format  to  minimize  the  possible  confounding  effect  of  writing  ability,  cadets  described 
a  recent  situation  they  experienced  that  elicited  behavior  relevant  to  the  performance  dimension 
of  interest  and  the  behavior  they  exhibited.  Trained  raters  used  standardized  rating  scales  to 
evaluate  the  level  of  skill  that  behavior  represents.  By  allowing  respondents  to  describe  their  best 
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example  of  behavior  relevant  to  a  performance  dimension,  we  can  assess  the  extent  to  which  the 
person  is  able  to  exhibit  the  behavior  irrespective  of  the  extent  to  which  they  typically  exhibit  the 
behavior.  To  measure  the  components  of  motivation  described  in  Johnson's  (2003)  model,  we 
created  (a)  a  unique  self-report  measure  of  work  habits;  (b)  a  motives  scale  assessing  attitudes, 
values,  and  preferences  that  predict  performance;  (c)  a  cognitive  processes  motivation  scale 
measuring  self-efficacy,  expectancy,  goal  content,  and  goal  commitment;  and  (d)  a  self-report 
measure  of  self-regulation  with  respect  to  targeted  behaviors. 

Potential  Army/Military  Applications 

Support  for  the  performance  model  will  provide  the  Army  with  a  tool  that  can  be  used  to 
identify  the  constructs  through  which  individual  difference  variables  work  to  influence 
performance  on  specific  dimensions.  This  is  especially  important  for  citizenship  and  adaptive 
performance,  which  are  important  components  of  Army  officer  jobs  but  have  received  little 
research  attention  in  a  military  or  civilian  context.  Such  a  tool  would  contribute  to  a  better 
understanding  of  the  relationships  between  predictors,  mediators,  and  job  performance  criteria. 
For  the  Army,  this  tool  would  probably  have  its  best  application  in  identifying  training  and/or 
development  needs.  Given  a  criterion  construct  on  which  an  individual's  performance  is  in  need 
of  improvement,  this  model  can  help  to  identify  the  determinants  of  performance  on  that 
construct.  For  example,  an  individual  possessing  adequate  skill  and  knowledge  may  determine 
that  he  or  she  must  learn  new  self-regulatory  strategies  to  overcome  counterproductive  task- 
related  habits. 

This  project  will  yield  several  products  that  may  be  very  useful  to  the  Army.  First,  the 
various  motivation-related  instruments  we  developed/assembled  and  will  validate  against  criteria 
of  importance  to  the  Army  should  prove  to  be  quite  useful  as  self-development  tools.  Completing 
these  instruments  can  help  Soldiers  and/or  officers  learn  about  their  own  attitudes,  values,  goal 
commitment,  self-efficacy,  self-regulatory  skills,  and  habits  relevant  to  five  critical  performance 
dimensions. 

Second,  the  Past  Behavior  Record  Form  may  prove  to  be  useful  for  training.  In  particular, 
the  evaluation  guide  used  to  score  this  instrument  will  provide  useful  information  regarding 
where  various  skill-relevant  behaviors  fall  on  various  performance  continua.  Moreover,  the 
principles  induced  during  the  process  of  calibrating  the  performance-relevant  skills  to  different 
points  on  these  anchored  rating  scales  will  make  useful  learning  points  in  training  programs.  The 
situational  judgment  test  will  also  be  a  useful  self-paced  training  tool.  Individuals  are  presented 
with  realistic  situations  and  choose  from  a  variety  of  alternative  response  options.  Explanations 
of  the  strengths  and  weaknesses  of  each  response  can  be  developed  to  help  the  trainee  understand 
the  level  of  effectiveness  of  the  chosen  option  and  the  most  effective  option. 

Finally,  the  performance  rating  form  may  have  applied  value  for  performance  appraisal, 
training  needs  analysis,  or  criteria  for  future  validation  studies.  In  addition,  it  would  make  a 
useful  feedback  and  development  tool.  The  performance  rating  form  is  a  multi-source  rating 
instrument.  As  such,  it  will  not  only  provide  a  profile  of  strengths  and  weaknesses,  it  will 
provide  information  about  discrepancies  between  how  an  individual  is  seen  from  different 
perspectives  (i.e.,  subordinates,  peers,  superiors).  It  would  be  extremely  useful,  for  example,  for 
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a  junior  officer  to  become  aware  of  differences  between  his  or  her  self-perceived  performance 
and  the  perceptions  of  his  or  her  performance  held  by  subordinates. 

Future  Plans 

By  the  end  of  Fiscal  2007,  we  will  conduct  the  following  activities: 

•  Complete  criterion  data  collection. 

•  Analyze  data.  Some  of  the  questions  to  be  addressed  with  these  data  are: 

o  Is  adaptive  performance  distinct  from  task  and  citizenship  performance? 

o  Which  SJT  scoring  method  yields  the  highest  validity  when  predicting 
performance? 

o  Does  this  SJT  measure  distinct  constructs? 

o  Is  each  instrument  developed  or  assembled  for  this  project  a  valid  predictor  of  the 
dimension(s)  of  performance  it  is  intended  to  predict? 

o  Which  type  of  rating  scale  developed  for  the  Past  Behavior  Record  yields  scores 
that  better  predict  performance? 

o  Do  the  data  fit  the  Johnson  (2003)  model,  in  terms  of  hypothesized  mediating 
variables  and  hypothesized  moderating  variables? 
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Problem(s)/Research  Question(s)  -  How  do  common  social  goals  relevant  to  military  situations 
(e.g.,  self-protection,  other-protection,  retribution)  influence  early-stage  cognitive  processing  of 
complex  social  environments?  To  which  person  in  a  crowd  will  we  pay  attention,  for  example, 
and  how  will  we  interpret  his  or  her  facial  expression?  How  do  these  goal-driven  processes  shape 
important  down-stream  behavioral  decisions  (e.g.,  to  shoot  vs.  not  shoot  a  possible  enemy 
combatant)? 

Technical  Barrier(s)  -  Until  recently,  little  effort  has  been  made  to  link  theory  about 
fundamental  social  goals  and  their  associated  emotions  to  theory  about  how  people  process 
information  during  the  first  seconds  of  an  initial  encounter. 

Significance/Impact  for  Basic  Research  -  This  research  will  help  conceptually  integrate  two 
important,  but  disconnected,  research  literatures  while  also  providing  empirical  data  that  tests 
hypotheses  about  goal  influences  on  cognitive  processing  of  complex  social  environments. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  for 
identifying  those  individuals  most  susceptible  or  resistant  to  particular  goal-related  information 
processing  biases,  identifying  those  individuals  most  in  need  of  training  to  overcome  such  biases, 
and  designing  training  programs  to  mitigate  such  biases.  We  suggest  that  our  research  may  be 
most  directly  relevant  to  the  following  applied  programs: 

•  The  Army's  Science  and  Technology  Objective:  Soldier  (STO  IV.SP.2002.01 ) — 

Selection  Classification,  and  Performance  Metrics  for  the  Future  Force  Soldier;; 

•  The  Army's  Science  and  Technology  Objective:  Training  (STO  IV. SP. 2003. 06) — 
Objective  Force  Warrior  Training; 

•  The  Army's  Science  and  Technology  Objective:  Training  (extension  of  Future  Force 
Warrior  Training) — Maximizing  Small  Unit  Performance; 

•  The  Army's  Science  and  Technology  Objective:  Training — Ground  Systems  Team 
Training; 

•  The  Army's  Science  and  Technology  Objective:  Training  (STO  IV. SP. 2003. 06) — 
VICTOR:  Virtual  Individual  and  Collective  Training  for  Future  Warriors;  and 

•  The  Army's  Science  and  Technology  Objective:  Leadership  Development — Assessing 
and  Developing  Leaders  for  Special  Operations  Units. 


Overview 


Soldiers  often  encounter  extremely  complex  social  situations  in  which  they  must  rapidly 
make  difficult,  life-or-death  decisions.  However,  human  cognitive  capacities  are  limited  in  ways 
that  make  it  impossible  to  attend  equally  to  every  individual  or  even  to  all  characteristics  of  any 
given  individual  encountered  in  such  a  setting  (imagine  a  crowded  market  in  a  large  city). 
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Instead,  people  must  focus  their  attentional  and  information  processing  resources  on  a  small 
subset  of  individuals  and  characteristics.  The  selective  direction  of  cognitive  processing  often 
occurs  automatically,  without  conscious  intent,  and  is  highly  susceptible  to  biases  that  can 
potentially  lead  to  serious  errors  (e.g.,  failing  to  process  clues  of  an  impending  fatal  ambush  or 
mistaking  an  innocent  civilian  for  an  armed  insurgent).  Furthermore,  a  person's  current 
motivational  state  may  change  the  ways  in  which  that  person  allocates  attention  and  other  limited 
cognitive  resources.  The  aim  of  the  proposed  research  is  to  assess  how  common  social  goals — 
self-protection,  other-protection,  and  retribution — shape  attention  and  information  processing  in 
complex  social  environments  and  thereby  influence  important  down-stream  behavioral  decisions. 

Research  Approach 

We  proposed  to  run  10  laboratory  experiments  designed  to  test  hypotheses  derived  from 
our  conceptual  framework.  Each  experiment  has  employed  established  research  paradigms  and 
methodologies  used  in  social  psychology  and  cognitive  science  designed  to  tap  different  levels  of 
processing — from  visual  attention  (measured  via  eye-tracking)  and  interpretation  through 
memory  and  decision-making  (e.g.,  tasks  requiring  rapid  friend/enemy  distinctions  and 
shoot/don’t-shoot  decisions).  In  addition,  each  study  has  included  manipulations  of  participants' 
goals  (self-protection,  other-protection,  and/or  retribution).  Besides  experimental  manipulations, 
we  have  assessed  chronic  individual  differences  in  these  goals.  Participants  are  confronted  with 
complex  social  scenes  in  which  target  individuals  differ  in  their  sex,  age  (combatant  age  or  not), 
group  membership  (ingroup,  outgroup,  or  outgroup  ally),  facial  expression  (angry  or  not),  and/or 
possession  of  a  weapon.  We  have  examined  how  these  different  goals  alter  visual  attention  to, 
encoding  of,  memory  for,  and/or  behavioral  decisions  about  the  different  individuals  in  the 
scenes. 

Accomplishments 

Thus  far,  we  have  completed  data  collection  for  all  1 0  proposed  experiments,  and  are 
currently  completing  data  analysis,  integrating  findings  across  studies,  and  beginning  to  prepare 
research  papers  describing  the  interesting  results.  To  overview,  preliminary  results  indicate 
distinctive  patterns  of  cognitive  bias  in  the  processing  of  adult  Arab  males,  as  well  as  several 
interesting  interactions  between  cognitive  processing  and  participants'  motivational  states. 
Participants  in  control  conditions  are  generally  not  good  at  distinguishing  adult  Arab  males  they 
have  seen  from  those  they  have  not  seen.  Furthermore,  adult  Arab  males  are  most  likely  to  be 
labeled  as  enemies  (even  when  they  are  not),  and  to  have  anger  attributed  to  them  (even  when 
they  are  displaying  neutral  facial  expressions).  When  any  one  of  various  forms  of  threat  are 
aroused,  people  become  better  at  distinguishing  among  individual  Arab  males.  Men  who  are 
experiencing  revenge  motivation,  although  also  more  attentive  to  individual  Arab  males,  are  also 
quicker  to  decide  to  shoot  those  individuals  in  a  rapid  decision-making  task. 

Two  of  our  studies  (Experiments  1  and  2)  were  designed  to  assess  how  social  motivations 
(i.e.,  feeling  self-protective,  other-protective,  or  vengeful)  would  influence  visual  attention  as 
measured  via  eye-tracker  technology.  In  those  studies,  we  recorded  how  participants  visually 
scanned  a  group  of  people  (including  men  and  women,  as  well  as  boys  and  girls,  who  were  Arab 
or  European,  and  wearing  angry  or  neutral  facial  expressions).  We  later  asked  participants  to 


75 


identify  particular  faces  that  had  been  in  those  arrays,  and  the  test  included  an  equal  number  of 
people  in  each  category  that  they  had  not  seen  before.  This  combination  of  tasks  allows  us  to 
examine  processing  efficiency,  or  "bang  for  the  attentional  buck."  by  calculating  the  relationship 
between  amount  of  time  spent  looking  at  a  particular  type  of  face  and  later  memory  for  faces  in 
that  category.  One  interesting  finding  from  that  research  is  that  women  concerned  with  self¬ 
protection  become  increasingly  efficient  at  processing  Arab  males. 

Two  of  the  studies  (Experiments  3  and  4  from  the  proposal)  were  designed  to  assess  how 
the  same  social  motivations  affect  rapid  categorization  of  other  people  as  enemies  or  allies.  Each 
participant  viewed  128  photos  varying  by  gender,  group  (European- American  vs.  apparently 
Arab),  age  (mid-20s  vs.  elementary-age  children),  and  facial  expression  (angry  vs.  neutral).  Half 
the  targets  were  randomly  labeled  with  an  ingroup  insignia,  and  the  others  were  randomly 
labeled  with  the  enemy  insignia.  Data  analysis  for  Experiemnt  3  suggests  that  participants  were 
generally  biased  toward  identifying  Arabs  as  enemies  and  Europeans  as  friends;  they  are  also 
biased  toward  identifying  females  as  friends.  When  concerned  with  self-protection,  the  bias  to 
identify  Arabs  as  enemies  increased.  Moreover,  participants  were  slower  to  react  to  angry  Arab 
faces,  and  were  especially  biased  to  call  them  enemies  even  when  they  were  not;  this  was 
particularly  the  case  for  participants  who  possessed  implicit  prejudices  against  Arabs  to  begin 
with.  In  contrast,  participants  were  quick  to  react  to  neutral  European  faces,  and  were  especially 
biased  to  call  them  friends  even  when  they  were  not;  this  was  particularly  the  case  for 
participants  who  are  dispositionally  inclined  to  see  the  world  as  a  dangerous  place.  Revenge 
motivations  also  seem  to  create  a  bias  to  identify  ingroup  members  as  friends  (even  when  they 
were  not);  this  bias  was  pronounced  among  participants  who  dispositionally  view  the  world  as  a 
dangerous  place.  A  manuscript  based  on  these  results  is  being  prepared. 

Two  studies  (Experiments  5  and  6  from  the  proposal)  were  designed  to  assess  how  social 
motivations  affect  attributions  of  anger  to  neutral  faces  of  different  groups.  The  most  interesting 
finding  from  this  research  is  that  attibutions  of  anger  are  higher  for  Arabs,  for  men,  and  for 
adults.  As  a  result,  judges  generally  tend  to  attribute  relatively  high  levels  of  anger  to  adult  Arab 
males  (even  when  they  are  in  fact  displaying  neutral  facial  expressions). 

Two  of  the  proposed  studies  (Experiments  7  and  8  from  the  proposal)  were  designed  to 
assess  how  the  presence  of  individuals  relevant  to  some  motives  might  interfere  with  the 
cognitive  processing  of  other  individuals  relevant  to  other  goals.  In  these  studies,  participants 
view  pairs  of  photographs  that  contain  either  an  adult  Arab  male,  an  adult  European-origin 
female,  or  a  crying  child,  to  examine  (1 )  whether  and  in  what  way  these  stimuli  interfere  with  the 
processing  of  other  faces  presented  alongside  them,  and  (2)  whether  the  nature  of  this 
interference  differs  depending  on  the  perceiver's  active  and  chronic  goals.  Whereas  participants 
had  an  especially  difficult  time  accurately  differentiating  Arab  targets  from  one  another  in 
memory  under  baseline  conditions,  each  of  the  threat  manipulations  improved  this  performance. 
Morever,  each  of  the  threat  motivations  created  a  tendency  for  Arab  male  targets  to  "trump"  the 
recognition  of  white  female  targets  (i.e.,  to  make  it  more  difficult  for  participants  to  differentiate 
in  memory  European-origin  females  from  one  another);  in  the  retribution  condition,  Arab  male 
targets  also  "trumped"  recognition  memory  for  (the  normally  attention-grabbing)  crying 
children.  For  our  participants,  the  presence  of  Arab  males  appeared  to  "steal"  processing  away 
from  other  target  individuals  in  the  social  context. 
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Two  of  the  studies  (Experiments  9  and  10  from  the  proposal)  were  designed  to  assess 
how  the  three  motivations  affect  rapid  categorization  of  other  people  as  threats  or  non-threats  and 
the  decision  to  shoot  those  carrying  a  weapon.  Each  participant  viewed  photos  varying  by 
gender,  group  (European- American  vs.  apparently  Arab),  and  facial  expression  (angry  vs. 
neutral).  Half  the  targets  were  holding  a  weapon,  and  the  other  half  were  holding  a  wallet  or  a 
cell  phone.  The  participant's  task  was  to  shoot  only  armed  targets,  regardless  of  gender, 
ethnicity,  or  emotion.  When  motivated  by  retribution,  participants  were  especially  likely  to  not 
shoot  unarmed  Europeans  and  to  shoot  armed  Arabs,  but  they  also  exhibited  a  bias  to  shoot 
unarmed  Arabs. 

Contributions  to  Basic  Research 

This  research  promises  to  make  several  important  contributions:  (1)  It  theoretically  links 
research  literatures  addressing  how  fundamental  human  goals  affect  information  processing  with 
ecologically  inspired  theories  of  motivation  and  cognition;  (2)  The  particular  findings  are 
contributing  to  a  theoretical  framework  for  understanding  how  goals  related  to  self-protection, 
other-protection,  and  retribution  shape  early-stage  attention  and  interpretation  of  complex, 
information-rich  social  environments,  and  thereby  influence  subsequent  behavioral  decision 
making;  (3)  We  have  been  able  to  provide  rigorous  empirical  tests  of  the  hypotheses  derived 
from  this  framework;  and  (4)  The  findings  have  suggested  a  number  of  interesting  new  questions 
worth  pursuing.  Thus,  this  research  should  help  expand  upon  the  field's  understanding  of 
motivated  social  perception  and  cognition. 

Potential  Army/Military  Applications 

Several  of  these  findings  seem  highly  relevant  to  understanding  how  a  Soldier's 
perceptual,  cognitive,  and  decision  making  processes  may  depend,  for  example,  on  whether  he  or 
she  is  primarily  concerned  in  the  moment  with  self-protection,  protection  of  a  friend  or  comrade, 
retribution  for  an  attack  on  one’s  country  or  the  death  a  comrade,  or  the  safety  and  well-being  of 
innocents  in  the  field.  Although  not  all  data  are  yet  analyzed,  initial  results  do  have  direct 
relevance  to  Soldiers  making  judgments  in  conflict  situations.  The  findings  that  judges  are 
generally  not  good  at  distinguishing  Arab  males  from  one  another,  are  disposed  to  label  those 
individuals  as  angry  and  as  enemies  even  when  they  are  not,  and  are  quicker  to  make  a  decision 
to  shoot  those  individuals  are  of  direct  relevance  to  Soldiers  involved  in  Middle  Eastern 
conflicts,  who  must  in  fact  distinguish  between  friendly  and  hostile  Arabs.  Given  that 
combatants  may  include  both  men  and  women,  it  is  also  of  interest  that  revenge  motivation 
increased  the  proclivity  of  men  to  shoot  Arab  males,  whereas  self-protection  increased  the 
proclivity  of  women  to  process  Arab  males  efficiently  and  accurately. 

Future  Plans 

Several  preliminary  findings  from  this  program  have  been  presented  at  national 
conferences,  and  are  currently  being  developed  into  manuscripts  for  publication.  We  also  are 
conducting  additional  analyses  of  the  sometimes  complex  data  sets,  such  as  linking  eye-tracking 
and  memory  findings,  as  well  as  exploring  individual  differences.  We  expect  these  more 
thorough  analyses  to  uncover  additional  interesting  patterns.  Several  potential  follow-up  studies 
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are  suggested  by  the  current  patterns  of  findings.  Possible  lines  of  follow-up  research  involve  the 
questions:  (1 )  Do  there  exist  individual  differences  in  the  patterns  of  these  cognitive  biases,  such 
that  individuals  may  be  characterized  by  “syndromes”  of  biases  they  exhibit,  perhaps  related  to 
chronic  sensitivities  to  fundamental  motives,  and  that  may  influence  how  these  individuals 
process  information  in  complex  social  environments?  Such  a  question  would  seem  to  have 
important  implications  for  how  one  might  identify  those  military  personnel  particularly 
susceptible  to  particular  decision  errors  (or  high  quality  decisions)  in  the  field.  One  approach  to 
exploring  this  would  be  to  administer  batteries  of  all  our  different  measures  and  tasks  to  the  same 
subjects  across  multiple  measurement  sessions,  thereby  enabling  us  to  identify  social-cognitive 
processing  "types.”  (2)  Given  these  demonstrable  biases,  and  their  particular  relationships  with 
motivational  states,  how  would  one  train  these  biases  away?  Would  the  training  approach  diiter 
depending  on  the  particular  patterns  (syndromes)  of  bias  individuals  exhibited?  (3)  The  types  of 
cognitive  biases  we've  demonstrated  here  also  will  likely  be  exhibited  by  those  in  contact  with 
U.S.  Soldiers,  such  as  civilians  and  enemy  combatants  in  foreign  lands.  Given  what  we're 
learning  about  how  the  biases  are  elicited,  and  their  motivational  underpinnings,  what  strategies 
might  be  useful  to  lessen  the  likelihood  or  reduce  the  impact  they  have  on  the  perceptions  and 
treatment  of  these  Soldiers? 
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An  Integration  of  Motivation  Theories 


Contract  #:  DASW01-04-K-0001  Contract  Dates:  12/12/2003  to  12/1 1/2007 

Institution:  The  Hebrew  University  of  Jerusalem  PI:  Avraham  N.  Kluger 

Liaisons:  Trueman  Tremble,  SARU 
Robert  Solick,  LDRU 


Problem(s)/Research  Question(s)  -  What  are  the  implications  of  win-win  approaches  to 
between-person  negotiations  for  inner  person  conflict  resolution? 

Technical  Barrier(s)  -  Studying  real  conflicts  without  upsetting  participants  and  helping  them 
in  return  for  their  participation. 

Significance/Impact  for  Basic  Research  -  Extending  our  tool  box,  which  we  started  to  develop 
in  2005  and  reported  in  2006  ( Feedforward ),  to  provide  tools  for  increasing  both  well-being  and 
productivity  of  employees,  coupled  with  testable  theoretical  rationale. 

Potential  Transitions  -  Knowledge  gained  during  our  2006  work  may  be  usefully  employed  in: 

•  Performance  Appraisal  ( Feedforward  reported  last  year); 

•  Various  Army  training  programs;  and 

•  LEADERSHIP:  Developing  leaders  in  a  changing  Army  (WP  103). 

Overview 

The  current  work  has  diversified  into  multiple  inter-related  projects.  Due  to  space 
limitations,  however,  the  report  below  focuses  on  the  key  development  during  2006 — The 
Negotiational  Self  (with  Ph.D.  student  Dina  Nir). 

Negotiation  is  a  fact  of  life.  We  negotiate  every  day  of  our  lives  and  in  every  possible 
kind  of  situation,  whether  we  are  trying  to  buy  a  house,  build  team  consensus,  decide  where  to  go 
for  lunch,  or  convince  the  kids  to  go  to  bed.  These  negotiations  call  for  the  unfolding  of  complex 
communication  processes  between  two  or  more  people  with  the  aim  of  reaching  an  agreement. 
Recent  developments  in  understanding  the  complex  concept  of  the  ^//"suggest  that  we  do  not 
only  negotiate  with  others,  but  first  and  foremost  we  negotiate  within  ourselves  (Hermans,  1996). 

The  self,  according  to  Hermans  (1996)  is  organized  as  a  dialogical  interchange  between 
mutually  influencing  selves,  or  1-positions  in  an  imaginary  landscape  of  the  mind.  These 
dialogical  interchanges  allow  communications  between  often  opposing  and  conflicting  I- 
positions  to  be  internally  voiced  so  that  a  decision  can  be  reached.  In  the  current  work,  internal 
dialogical  interchanges  are  viewed  as  intra-personal  negotiations  that  guide  and  shape  one  s 
choice  and  behavior;  hence  the  term  negotiational  self  is  introduced.  According  to  this  stance, 
negotiations  within  the  self  correspond  to  inter-personal  negotiation  processes  and  strategies,  and 
therefore  can  be  explained,  analyzed  and  ultimately  transformed  through  applying  collaborative 
principles ,  developed  in  the  field  of  negotiation. 
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In  negotiations,  reaching  integrative  solutions,  as  opposed  to  reaching  distributive 
outcomes  or  compromises,  can  be  facilitated  by  using  principled  negotiation,  a  term  given  to  the 
interest-based  approach  to  negotiation  first  formulated  by  Fisher  &  Ury  (1981).  The  core  of  this 
approach,  which  is  considered  the  “big  bang  theory”  in  negotiation  (Thompson  &  Leonardelli, 
2004),  is  that  understanding  one's  deep-seated  needs  and  interests,  rather  than  one’s  positions,  is 
the  key  to  creating  innovative  and  creative  solutions,  and  reaching  integrative  agreements. 
Interests  are  distinct  from  positions  in  that  interests  are  the  underlying  concerns,  needs,  desires, 
or  fears  that  motivate  a  negotiator  to  take  a  particular  position  (Fisher,  Ury,  &  Patton,  1991 ). 
Positions,  on  the  other  hand,  are  the  specific  and  tangible  items  people  say  they  want.  When 
analyzing  the  interest,  it  is  also  important  to  distinguish  between  visible  and  hidden  interests. 
Visible  interests  are  those  interests  that  people  are  willing  to  talk  about  and  discuss  openly.  In 
contrast,  hidden  interests  are  often  personal  interests  that  people  tend  to  avoid  discussing,  such  as 
increasing  status,  power,  recognition  and  esteem.  However,  more  often  than  not,  these  hidden 
interests  have  a  decisive  effect  on  organizational  decisions  and  outcomes. 

Armed  with  these  insights  regarding  interpersonal  negotiations,  four  hypotheses  are  put 
forward:  First,  as  in  inter-personal  negotiations,  negotiations  within  the  self  lead  to  either 
integrative  (win-win),  or  distributive  (win-lose)  outcomes.  Second,  each  of  these  outcomes  (win- 
win  or  win-lose)  leaves  its  unique  affective  trace.  Third,  people  frequently  use  either  an 
integrative  or  distributive  negoliational  schema  that  manifests  itself  in  both  intra-personal  and 
inter-personal  negotiations.  Finally,  inner  negotiations  can  be  transformed  from  distributive  to 
integrative  processes  with  an  interest-based  approach  to  negotiations.  An  interest-based  approach 
is  expected  to  promote  innovative  solutions  that  satisfy  conflicting  needs  and  interests  within  the 
self.  Two  empirical  studies  were  already  run  to  address  and  validate  this  theoretical 
development.  In  the  first  study,  a  convenience  sample  of  99  respondents  answered  an  open- 
ended  survey  regarding  one  key  conflict  from  their  past.  This  study  (see  below)  allowed  us  to 
establish  that  (a)  inner  conflicts  can  be  reliably  classified  on  a  scale  ranging  from  win-lose  to 
win-win,  just  like  between-person  conflicts,  and  (b)  emotions  reflect  the  quality  of  the  decision. 

In  a  second  study  (data  are  still  being  collected),  respondents  reported  an  inner  conflict  and  their 
current  solution  for  it.  They  were  given  instruction  regard  integrative  negotiation,  asked  to  offer 
a  new  inner  conflict  solution,  asked  to  discuss  their  solution  with  a  supportive  but  challenging 
partner,  and  to  report  their  new  solution  again.  Initial  results  suggest  that  this  training  enhanced 
the  degree  to  which  one's  conflict  resolution  reached  a  win-win  solution  for  his  or  her  multiple 
inner  voices. 

Research  Approach 

In  addition  to  the  theoretical  development  that  was  based  on  multiple  meetings  with 
Ph.D.  students,  and  clinical  observations  gained  while  coaching  multiple  managers  with  these 
ideas,  here  I  report  the  approach  of  the  first  completed  study.  A  convenience  sample  of  99 
respondents  answered  a  questionnaire  regarding  one's  salient  but  solved  conflict.  Next 
respondents  were  instructed  to  list  all  the  voices  they  considered  in  making  the  decision,  share 
their  actual  decision,  and  report  their  emotions  at  the  time  of  the  decision.  A  scale  of  emotion 
ratio  was  calculated  by  dividing  the  mean  of  the  positive  emotions  items  by  the  mean  of  the 
negative  emotion  items  (a  score  larger  than  1  means  largely  positive  emotions).  The  conflict 
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scripts  were  read  by  two  judges  who  independently  rated  them  on  a  -5  (win-lose)  to  +5  (win-win) 
scale. 

Accomplishments 

First,  an  inter-rater  reliability  (1CC2)  of  the  two  judges  was  .80,  indicating  that 
negotiation  outcomes  within  the  self  can  be  reliably  identified  on  a  continuum  ranging  from  win- 
lose  to  win-win. 

Second,  the  correlation  between  type  of  outcome  (win-win  or  win-lose),  and  the  ratio  of 
positive  over  negative  emotions  was  high:  r  =.51.  However,  inspecting  the  scatter  plot  of  these 
data  (Figure  1)  is  even  more  instructive. 
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Figure  1.  Emotions  and  internal  negotiation  outcomes. 

The  blue  dots  represent  individuals’  scores  both  on  the  judges'  scale  of  outcome  (the  X 
axis)  and  their  own  self  report  regarding  their  emotions  (the  Y  axis).  The  solid  line  represents  the 
prediction  of  one’s  emotions  on  the  basis  of  the  judges’  rating  of  one’s  outcomes  (the  regression 
line).  The  dotted  line  marks  emotion  ratio  of  1.  Scores  higher  than  1  indicate  largely  positive 
emotions.  Two  features  of  this  scatter  plot  are  striking.  First,  the  majority  of  respondents  (over 
2/3)  reported  reaching  win-lose  outcomes.  Second,  only  one  person  reported  largely  negative 
emotions  and  was  rated  to  have  a  win-win  outcome,  in  contrast  to  14  respondents  who  reported 
largely  negative  emotions  and  were  rated  to  have  a  win-lose  outcome  (compare  the  number  of 
dots  below  the  dotted  line  to  the  right  of  0  on  the  X  axis  with  number  of  dots  to  the  left  of  the  0). 
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This  suggests  that  emotions  reflect  the  quality  of  the  negotiation  outcomes  within  the 
self:  The  more  win-win  one's  outcome,  the  better  one  feels.  This  may  appear  trivial,  but  suggests 
the  following  intervention  whenever  a  person  reaches  a  decision,  ask  “How  do  you  feel  now?"  If 
you  do  not  feel  well,  you  may  want  to  review  your  decision  and  search  for  a  better  solution. 

Contributions  to  Basic  Research 

First,  our  findings  indicate  that  the  literature  on  interpersonal  negotiations  can  be 
integrated  with  the  literature  on  the  multi-voiced  self  to  expand  personnel  and  leadership 
development.  Specifically,  tools  used  to  separate  interests  from  position  can  be  relevant  for 
person  development.  Second,  our  findings  hint  at  the  possibility  that  most  people  reach  sub- 
optimal  decisions,  due  to  an  inner  schema  of  win-lose.  This  opens  the  door  for  multiple 
theoretical  questions  and  practical  interventions.  For  example,  is  the  tendency  to  reach  win-win 
solutions  chronic?  If  yes,  it  may  have  implications  for  personnel  selection.  Finally,  if 
corroborated,  our  findings  suggest  a  way  of  understanding  emotions  as  a  reflection  of  the  quality 
of  the  inner-theatre.  Again,  this  has  both  theoretical  and  applied  implications. 

Potential  Armv/Military  Applications 

We  believe  that  training  in  inner-conflict  resolutions  can  be  applied  in  various  military 
settings.  At  this  level  of  development  of  our  negotiational  self  tools,  we  believe  it  can  be  tried  in: 

•  Designing  training 

•  Leadership  development 

Future  Plans 

The  plan  for  2007 — the  last  year  of  this  contract — is  to  complete  the  current  and  two 
other  key  projects.  First,  it  is  planned  to  test  the  effectiveness  of  training  people  to  think  win-win 
while  solving  inner  conflicts.  We  also  hope  to  test  whether  inner  conflict  resolution  style 
influences  between-people  negotiation  style. 

Second,  the  feedforward  tool  reported  in  2006  is  now  under  empirical  testing.  The  plan  is 
to  demonstrate  that  the  feedforward  interview  yields  more  positive  emotions,  more  positive 
social  emotions,  better  insights,  and  better  performance  in  comparison  to  feedback  and  controls. 

Third,  another  project  to  be  fully  reported  next  year  is  geared  towards  integrating  goal- 
orientation  theory  with  value  theory.  Its  essence  is  showing  that  current  thinking  about 
achievement  situations  can  be  augmented  by  considering  amity  goals.  It  is  planned  to  complete 
(a)  empirical  demonstration  that  amity  goals  are  relevant  and  influential  in  achievement 
situations,  and  (b)  experiments  demonstrating  that  coupling  amity  goals  with  mastery  goals  (as 
opposed  to  contest  goals  with  performance  goals)  insulate  people  from  the  damaging  effects  of 
negative  feedback. 
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Comprehension  and  Memory  of  Spatial  and  Temporal  Event  Components 


Contract  #:  DASW01-02-K-0003 
Institution:  University  of  Notre  Dame 


Contract  Dates:  05/01/2002  to  04/30/2006 
PI:  Gabriel  A.  Radvansky 
Liaison:  Tonia  Heffner 


Problem(s)/Research  Question(s)  -  How  do  people  detect  and  process  changes  in  space  and 
time  for  on-going  situations?  That  is,  how  is  effective  cognition  influenced  by  the  environmental 
structure  of  dynamic  events  as  they  are  happening? 

Technical  Barrier(s)  -  Models  and  theories  of  event  comprehension  and  memory  are  only 
currently  emerging  and  virtual  reality  technologies  are  also  developing. 

Significance/Impact  for  Basic  Research  -  Understanding  how  people  monitor  and  update 
changes  in  situations  will  allow  for  more  developed  models  of  how  people  comprehend  and 
remember  information  about  unfolding  events  in  the  world. 

Potential  Transitions  -  Knowledge  gained  in  this  research  may  be  usefully  employed  in: 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  211); 

•  MEASURE  DIGITAL:  Defining  and  Measuring  Digital  Skill  Proficiency  (WP  234); 

•  Future  Force  Warrior  Training  ( WP  215);  and 

•  VICTOR:  Virtual  Individual  and  Collective  Training  for  Future  Warriors  (WP  233). 


Overview 


To  perform  successfully,  people  need  to  deal  with  changes  that  occur  during  events.  They 
need  to  identify  when  changes  have  occurred,  and  update  their  understanding  of  the  event 
accordingly. 

There  are  many  components  of  event  memory  and  comprehension  (Zwaan  &  Radvansky, 
1998),  including  space,  time,  entities,  causality,  and  intentions.  Recent  work  by  Radvansky  and 
Copeland  (2000;  2001;  2004a  2004b;  Radvansky,  Copeland,  Berish  &  Dijkstra,  2003; 

Radvansky,  Copeland  &  Zwaan,  2003;  Radvansky,  Zwaan,  Curiel  &  Copeland,  2001)  has  shown 
that  the  ability  to  track  these  changes  is  unrelated  to  working  memory  span  (i.e.,  the  amount  of 
information  that  a  person  can  hold  in  mind  at  one  time).  Instead,  it  appears  to  involve  mental 
processes  that  are  specific  to  situations  and  events. 

•  The  focus  of  this  project  is  on  changes  in  event  space  and  time  when  people  need  to 
know  whether  elements  from  the  previous  situation  are  still  relevant.  For  example,  for 
learning  about  a  traveler  going  from  New  York  to  London,  a  person  needs  to  mentally 
track  that  the  traveler  brought  a  computer  disk  with  classified  information  with  her,  and 
left  behind  a  book  of  contacts.  Although  space  and  time  are  basic  elements  of  events, 
people  do  not  always  actively  monitor  and  update  their  understanding  (Zwaan  &  von 
Oostendorp,  1993;  Zwaan,  Radvansky,  Hilliard,  &  Curiel,  1998). 
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•  More  recently,  we  have  extended  our  efforts  beyond  language  comprehension  to 

interactive  experiences  in  virtual  reality.  We  assess  how  spatial  updating  influences  the 
actual  experiences  with  events.  This  line  of  work  addresses  two  issues. 

o  First,  when  there  are  changes  in  spatial  location,  information  about  entities  carried 
over  to  the  new  location  should  remain  available  in  memory,  whereas  information 
about  entities  that  are  left  behind  should  be  less  available  (e.g.,  Glenberg,  Meyer, 
&  Lindem,  1987). 

o  The  second  is  that  as  a  person  moves  through  space,  the  further  things  are  from 
the  current  location,  the  less  available  that  information  should  be  (Morrow, 
Greenspan  &  Bower,  1987).  By  looking  at  how  changes  in  space  during  an  on¬ 
going  event  are  disrupting  thinking,  we  can  target  those  aspects  of  the  situation 
that  are  giving  people  the  most  trouble,  allowing  training  and  technology  to  target 
these  deficits. 

Research  Approach 

In  some  of  our  experiments,  people  read  narratives,  one  sentence  at  a  time,  and  reading 
times  were  recorded.  Reading  times  provide  an  index  of  difficulty.  When  an  event  shift  occurs, 
people  exert  effort  to  update  their  understanding  of  the  situation,  and  reading  times  increase. 
Thus,  a  noticed  change  in  space  or  time  would  show  a  slow-down  in  reading  times.  In  addition, 
reading  was  interrupted  with  probe  words.  These  probe  words  assessed  the  progress  of  a  person 
mentally  updating  their  event  understanding  (i.e.,  WHAT  they  were  thinking  about).  The  task 
was  to  indicate  whether  these  words  referred  to  the  narrative.  People  would  repond  faster  when 
they  referred  to  elements  that  were  part  their  thinking  about  the  current  situation. 

In  our  virtual  reality  work,  people  navigated  through  environments  with  some  goal,  such 
as  moving  objects  from  room  to  room.  We  looked  at  the  availability  of  information  about  an 
object  as  a  function  of  whether  it  was  being  carried,  or  had  just  been  put  down.  This  assessment 
was  done  by  stopping  the  movement  and  presenting  people  memory  probes  (object  names).  The 
task  was  to  indicate  whether  the  object  was  one  that  was  currently  being  carried,  was  just  set 
down,  or  some  other  object.  In  other  cases,  people  memorized  the  layout  of  a  building  prior  to 
navigation.  At  different  points,  they  made  location  judgments  about  various  objects.  This  was 
done  to  assess  the  influence  of  both  prior  knowledge  and  spatial  distance  on  event  knowledge. 

Accomplishments 

Updating.  There  were  a  number  of  key  findings  from  this  research.  First,  people  are 
successful  at  updating  their  understanding  of  an  event  when  there  is  a  change  in  space  or  time. 
Second,  when  there  is  a  shift  to  a  new  spatial  location,  people  are  more  facile  at  making  this 
transition,  whereas  shifts  in  time  require  more  effort.  Third,  the  ease  with  which  spatial  shifts  are 
made  is  supported  by  the  finding  that  people  are  able  to  remove  information  about  newly 
irrelevant  entities  from  the  prior  situation  from  their  current  stream  of  thought.  Fourth,  the 
difficulty  people  have  with  a  shift  to  a  new  time  frame  is  supported  by  a  decreased  ability  to 
remove  information  about  newly  irrelevant  objects. 
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Virtual  Reality  (VR).  In  our  virtual  reality  studies  assessing  spatial  updating,  there  are  two 
aspects  of  event  interaction  that  impacts  the  ability  to  monitor  the  situation.  First,  knowledge  of 
things  that  people  are  carrying  is  more  available  than  things  they  have  just  set  down.  More 
importantly,  we  have  found  that  when  people  move  from  one  location  (room)  to  another, 
thinking  about  the  on-going  event  is  disrupted.  Most  prominently,  knowledge  of  what  objects  a 
person  is  carrying  is  disrupted.  This  is  shown  in  the  Figure  1  in  which  the  speed  that  people 
retrieve  information  is  slowed  after  there  is  a  spatial  shift. 


Figure  1.  Response  times  (in  ms)  to  object  name  probes  as  a  function  of  whether  people  had  just 
made  a  spatial  shift. 

Our  VR  studies  have  also  found  that  when  people  move  from  one  room  to  the  next  in  a 
well-known  area,  information  about  the  room  just  left  drops  in  availability  as  people  mentally 
work  to  understand  the  new  situation.  Figure  2  shows  the  speed  to  identify  information  about 
objects  when  a  person  has  moved  from  one  room  ( Source )  through  a  second  {Path)  and  ending 
up  in  a  final  (Goal)  room.  The  figure  also  shows  the  availability  of  objects  from  Other  rooms. 
People  are  slowest  at  retrieving  information  about  objects  from  a  room  they  passed  through, 
suggesting  that  shifts  in  location  disrupt  the  ability  to  retrieve  recent  event  information. 
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Figure  2.  Response  times  to  memory  probes  (object  name  pairs)  as  a  function  of  distance  from 
the  subjects'  current  position  in  the  virtual  environment. 
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Contributions  to  Basic  Research 


This  project  provides  a  more  advanced  understanding  of  how  people  monitor  event 
changes  in  the  world.  This  research  follows  several  recent  developments  in  theories  of  mental 
models  (mental  simulations  that  we  create)  and  theories  of  embodied  cognition  (how  our 
thinking  is  influenced  by  how  we  interact  with  the  world).  Our  analyses  will  allow  for  a  clearer 
understanding  of  how  people  comprehend  and  remember  changes  in  events.  From  these  data,  we 
hope  to  develop  a  means  of  identifying  when  people  are  having  trouble  updating  event 
information,  and  provide  a  more  targeted  means  of  dealing  with  those  difficulties. 

Potential  Army/Military  Applications 

The  potential  applications  of  this  research  for  the  Army  are  in  the  areas  of  training  and 
system  interfaces.  In  the  area  of  training,  this  research  will  provide  insight  into  the  specific 
difficulties  people  have  updating  their  understanding  of  events.  Knowing  in  what  ways  people 
have  trouble  tracking  changes  in  space  and  time  will  allow  for  the  development  of  procedures 
that  would  directly  target  such  difficulties.  In  addition,  because  the  modem  Soldier  needs  to 
interact  with  sophisticated  technologies  that  involve  tracking  various  entities  through  time  and 
space  (whether  they  be  enemy  forces,  one’s  own  supplies,  friendly  forces,  noncombatants,  etc.), 
knowing  the  capabilities  and  limits  of  human  event  comprehension  and  memory  will  help  in  the 
design  of  these  systems.  Spatial  and  temporal  information  that  is  readily  tracked  by  people  would 
need  either  minimal  or  no  feedback  from  the  system.  However,  those  aspects  of  spatial  and 
temporal  processing  that  are  more  difficult  would  be  a  higher  priority  for  that  system,  in  terms  of 
augmenting  the  Soldier's  current  capabilities. 

Final  Summary 

The  aim  of  this  research  was  to  better  understand  the  influences  of  the  structure  of  events 
in  the  world  on  the  ability  of  people  to  comprehend,  remember,  and  generally  process 
information  in  ongoing  situations.  This  line  of  study  was  pursued  because  our  ability  to  manage 
knowledge,  and  hence  perform,  in  the  world  may  be  directly  influenced  by  how  we  interact  with 
the  world,  and  not  simply  be  the  result  of  our  intellectual  abilities  and  histories. 

To  explore  this  sort  of  event-based  cognition,  we  assessed  performance  when  people 
were  either  reading  texts  or  navigating  in  virtual  environments.  When  people  read  texts,  we 
assessed  performance  as  a  function  of  whether  a  shift  to  a  new  spatial  or  temporal  framework 
had  occurred  in  the  events  being  described.  We  measured  the  effect  of  that  shift  on  performance, 
as  well  as  the  ability  of  people  to  maintain  currently  relevant  information,  and  to  remove  newly 
irrelevant  information.  What  was  found  was  that  while  changes  in  spatial  location  are  more 
easily  processed,  people  have  difficulty  processing  information  about  temporal  shifts,  both  in 
terms  of  the  processing  of  the  change  in  the  time  frame,  as  well  as  the  coordination  of 
accompanying  knowledge  that  would  be  affected  by  the  temporal  shift. 

When  people  navigated  through  virtual  environments,  in  one  series  of  studies  we  had 
people  move  objects  about  and  tested  the  availability  of  knowledge  about  those  objects.  Of 
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critical  importance  was  whether  people  had  moved  through  a  doorway  or  not.  What  was 
observed  was  that  performance  declined  substantially  when  people  had  moved  from  one  location 
to  another. 

In  another  series  of  experiments  using  virtual  environments,  people  first  memorized  a 
map  of  the  larger  location  (a  building).  Then  they  navigated  through  the  building.  At  different 
points  people  were  stopped  and  tested  for  their  knowledge  of  objects  in  that  space.  What  was 
found  was  that  information  about  objects  in  rooms  that  were  passed  through  was  less  available 
than  information  about  objects  in  other  rooms.  In  essence,  people  were  actively  suppressing 
information  that  was  more  salient  in  the  environment,  but  not  relevant  to  the  current  task. 

The  key  findings  from  this  research  are:  (a)  that  the  ability  of  people  to  update  the 
knowledge  when  there  has  been  a  change  in  spatial  or  temporal  location  can  disrupt  thought 
processes,  (b)  it  is  more  difficult  for  people  to  adapt  to  changes  in  the  time  frame,  (c)  people  may 
inappropriately  maintain  information  after  a  change  in  time  period  occurs,  (d)  walking  through 
doorways  causes  forgetting,  (e)  passing  through  areas  without  interacting  with  objects  renders 
knowledge  of  those  objects  less  available,  and  (f)  the  effectiveness  of  thinking  is  directly 
influenced  by  the  structure  of  the  environment  and  how  a  person  is  interacting  with  that 
environment.  This  is  important  for  the  Army  because  Soldiers  and  units  are  more  often  required 
to  operate  in  complex  environments,  such  as  cities  and  large  buildings.  An  important  component 
of  improving  performance  in  these  settings  is  to  have  a  clear  understanding  of  how  the  structure 
of  the  operational  environment  influences  the  ability  to  perform  more  optimally,  thereby 
providing  the  opportunity  for  developing  methods  of  directly  addressing  these  cognitive 
challenges.  As  such,  it  is  recommended  that  Army  training  involve  relatively  complex 
environments  that  require  a  Soldier  to  monitor  multiple  sources  of  information  to  help  develop 
skills  needed  to  manage  that  information  in  relative  short  time  periods.  Moreover,  augmented 
cognition  technologies  that  are  developed  can  take  into  account  that  Soldiers  may  need  to  be 
reminded  of  mission  elements  when  there  have  been  large  transitions  in  the  ongoing  event 
structure. 

Future  research  on  these  issues  is  needed  to  explore  the  range  of  knowledge  that  is 
affected  by  event  transitions,  their  duration,  and  the  response  of  people  to  experiences  and 
training  in  circumstances  where  they  need  to  manage  knowledge  in  rapidly  changing  dynamic 
environments.  For  example,  it  is  important  for  Soldiers  to  have  an  accurate  working 
understanding  of  the  larger  battlefield,  other  than  their  own  particular  circumstances,  as  well  as 
understanding  their  role  within  that  larger  battlefield  space.  The  current  research  suggests  that 
the  updating  of  one’s  understanding  of  the  dynamic  environment  may  be  compromised  by  the 
actual  movement  through  it.  The  extent  and  duration  of  this  disruption  and  the  means  of 
diminishing  it  are  not  known  at  this  point.  Further  work  would  be  aimed  at  developing  a  clearer 
understanding  of  this  intercept  between  the  architecture  of  cognition  with  the  architecture  of  the 
world,  and  how  this  knowledge  can  be  leveraged  to  improve  human  performance  in  complex 
environments. 
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Social  Structures  Affecting  Army  Performance 


Contract  #:  DASW01-00-K.-0016 
Institution:  The  University  of  Maryland 


Contract  Dates:  08/17/2000  to  12/31/2004 

PI:  David  R.  Segal 

Co-PI:  Mady  Wechsler  Segal 


Problem(s)/Research  Question(s)  -  What  are  the  social  structures,  social  processes,  and  social 
psychological  characteristics  that  lead  young  Americans  to  enter  the  armed  forces,  and  affect 
their  morale,  satisfaction  with  service,  performance,  and  retention? 

Technical  Barriers  -  Social  structures,  social  processes,  and  social  attitudes  are  in  a  constant 
state  of  dynamic  change.  Social  institutions,  such  as  the  Army,  and  the  people  who  work  in  them 
must  constantly  adapt  to  changes  in  their  external  environment  and  in  their  own  organization. 

Significance/Impact  for  Basic  Research  -  Identification  of  important  dimensions  of  social 
change  and  how  people  and  organizations  respond  to  them  will  contribute  to  more 
comprehensive  theories  of  human  and  organizational  performance. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Army  accession  programs; 

•  Personnel  retention  programs; 

•  Organizational  transformation  programs;  and 

•  Human  resource  management  in  the  interest  of  maximizing  combat  performance. 


Overview 


One  of  the  major  trends  affecting  modem  American  organizations  is  outsourcing:  having 
functions  necessary  for  organizational  functioning,  but  not  core  to  the  mission  of  the 
organization,  performed  by  people  other  than  traditional  organization  members.  This  process, 
widely  practiced  by  American  corporations,  has  been  increasingly  adopted  by  the  federal 
government,  which  uses  contract  workers  who  are  not  federal  employees  to  perform  a  range  of 
functions.  Its  adoption  by  the  armed  forces  potentially  frees  a  larger  proportion  of  armed  forces 
personnel  for  core  military  tasks.  This  requires  consideration  of  the  issue  of  what  are  the  core 
military  functions  that  must  be  performed  by  uniformed  personnel. 

The  application  of  this  practice  to  the  armed  forces,  which  have  unique  employment 
characteristics,  has  two  major  manifestations:  basic  civilianization,  in  which  functions  formerly 
performed  by  military  personnel  are  now  performed  by  civilians  directly  employed  by  the 
service,  and  true  outsourcing,  in  which  functions  formerly  performed  by  military  personnel  or  by 
civilian  employees  of  the  military  departments  are  now  performed  by  civilians  who  are 
employees  of  a  civilian  corporation  that  has  contracted  with  the  service  to  do  the  work. 

Both  variants  of  civilianization  put  military  personnel  and  civilian  workers  in  shared 
work  environments,  where  they  may  perform  similar  or  complementary  tasks  but  work  under 
very  different  employment  conditions  (compensation,  housing,  discipline,  etc.).  Social 
comparison  processes  and  awareness  of  the  differences  in  conditions  of  employment,  which  tend 
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to  favor  the  civilians,  may  produce  feelings  of  relative  deprivation  among  military  personnel, 
which  in  turn  may  have  an  effect  on  their  job  satisfaction,  their  commitment  to  the  organization, 
and  their  intention  to  remain  in  service.  As  an  alternative,  they  may  elect  to  leave  the  service  at 
the  completion  of  their  obligation  and  enter  the  civilian  labor  force,  potentially  in  the  kinds  of 
jobs,  and  with  the  conditions  of  employment,  that  they  see  being  enjoyed  by  the  civilians  with 
whom  they  work. 

Awareness  of  the  differences  in  employment  conditions  on  the  part  of  the  civilians  may 
likewise  affect  their  willingness  to  continue  their  employment.  It  is  possible  that  the  ratio  of 
civilians  to  military  personnel  in  the  workplace,  and  the  frequency  or  intensity  of  interaction, 
have  implications  for  these  outcomes.  We  have  been  able  to  identify  no  previous  research  on  the 
social  psychological  consequences  of  military/civilian  workforce  integration  in  military  units. 

Research  Approach 

To  explore  these  issues,  we  conducted  field  research  at  two  military  work  sites  at  which 
military  personnel  interact  regularly  with  civilians  who  perform  support  functions:  an  Army 
combat  aviation  unit  located  outside  the  continental  United  States,  and  a  Navy  command  and 
control  ship  in  the  Pacific  Fleet.  While  in  both  cases  the  civilians  were  critical  to  the  overall 
functioning  and  mission  of  their  units,  the  Army  and  Navy  incorporated  their  respective  civilians 
in  different  ways.  The  civilians  working  on  the  Navy  ship  were  federal  employees,  while  those 
working  with  the  Army  aviation  squadron  were  civilian  contractors.  With  rare  exceptions,  the 
Army’s  civilian  contractors  had  been  brought  in  to  work  alongside  the  Soldiers,  frequently 
performing  the  same  tasks.  By  contrast,  the  civilian  mariners  with  the  Navy  performed  tasks  that 
had  been  categorically  civilianized,  so  there  were  no  sailors  performing  the  same  duties  as  the 
civilians.  We  conducted  surveys  of  the  military  and  civilian  personnel  at  both  work  sites.  In 
addition,  informal  interviews  were  conducted  with  civilians  and  military  personnel  at  both  sites. 
Most  of  these  interviews  were  individual  one-on-one  discussions,  but  occasionally  the  logistics 
of  the  situation  required  interviewing  groups  of  two  to  five  people. 

Accomplishments 

Both  our  surveys  and  our  interviews  show  that  Soldiers  and  sailors  in  our  investigation 
feel  relatively  deprived  compared  to  the  civilians  with  whom  they  work  as  a  result  of  the  social 
comparisons  they  make.  For  sailors,  these  feelings  of  relative  deprivation  decrease  with 
increased  levels  of  contact  with  civilians.  This  decrease  was  not  observed  among  Soldiers.  The 
civilians  and  military  personnel  report  being  satisfied  with  their  jobs,  but  the  civilians  were 
significantly  more  satisfied  than  the  Soldiers  and  sailors.  The  civilians  were  significantly  more 
committed  to  their  employers  than  the  Soldiers  and  sailors  were  to  the  military.  These  sentiments 
had  implications  for  people’s  intensions  to  remain  in  their  current  jobs.  While  at  least  three- 
quarters  of  the  civilians  at  both  sites  leaned  toward  or  planned  to  stay  with  their  current 
employer,  just  over  one-third  of  the  armed  forces  personnel  expressed  positive  intentions  to 
remain  in  the  military.  As  Figure  1  shows,  on  the  Navy  ship,  69%  of  the  civilians  and  only  29% 
of  the  sailors  planned  to  remain  in  their  current  job,  while  3 1  %  of  the  sailors  and  only  5%  of  the 
civilians  planned  to  leave.  Somewhat  fewer  of  the  Army  personnel  and  their  supporting  civilians 
were  certain  of  their  intentions,  as  reflected  in  Figure  2. 
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Retention  Intentions 


Figure  1.  Percent  frequency  distribution  of  Sailors’  and  CIVMARs'  retention  intentions. 

Social  comparisons  significantly  and  negatively  impact  Soldiers'  and  sailors'  intentions 
to  remain  in  military  service  beyond  current  enlistment  obligations.  But  this  effect  is  indirect, 
operating  through  job  satisfaction  and  organizational  commitment,  which  in  turn  affect  plans  to 
stay  in  the  service.  Social  comparisons  were  not  as  important  to  the  civilians'  plans  to  stay  in 
their  current  jobs. 
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Figure  2.  Percent  frequency  distribution  of  Soldiers'  and  civilian  contractors’  retention  intentions. 

Contributions  to  Basic  Research 

This  project  provides  insights  into  the  social  comparison  processes  that  take  place  when 
people  working  under  very  different  conditions  of  employment  share  a  common  work  space.  It 
contributes  to  our  understanding  of  how  feelings  of  relative  deprivation  develop,  how  they 
impact  on  work-related  attitudes  such  as  job  satisfaction  and  organizational  commitment,  and 
how  these,  in  turn,  may  contribute  to  turnover  or  retention  in  organizations.  More  generally,  it 
helps  us  to  understand  how  the  structure  of  an  organization  affects  those  who  work  there. 

Potential  Military  Applications 

Outsourcing  has  been  a  major  trend  in  American  industry,  and  a  great  deal  of  this  human 
resource  technology  has  been  transferred  to  the  federal  government,  including  the  Department  of 
Defense.  In  the  American  armed  forces,  it  has  been  accompanied  by  an  ongoing  civilianization 
of  functions  previously  performed  by  military  personnel,  and  these  processes  have  accelerated 
since  the  advent  of  the  Global  War  on  Terrorism.  Much  of  this  process  has  focused  on  potential 
dollar  savings  and  has  taken  place  without  regard  to  the  social,  psychological,  and  behavioral 
consequences  of  outsourcing  and  civilianization.  This  research  gives  human  resource  managers 
in  the  military  some  basis  for  evaluating  the  potential  personnel  costs  and  benefits  of  these 
processes. 
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Final  Summary 

This  analysis,  and  the  broader  project  of  which  it  is  a  part,  looks  at  the  social  institutions, 
attitudes,  and  values  that  lead  people  to  select  different  post-secondary  school  alternatives 
(military  service,  civilian  employment,  higher  education),  and  for  those  who  select  military 
service,  the  processes  that  influence  retention  and  departure.  Earlier  research  in  this  program 
identified  declines  in  enlistment  propensity  and  increasing  perceptions  that  the  military 
discriminated  on  the  basis  of  gender  and  race  and  was  not  a  family-friendly  employer.  These 
findings  presaged  an  actual  decline  in  enlistment  and  difficulties  in  recruitment.  The  current 
stage  of  research  suggests  that  the  processes  of  civilianization  and  outsourcing  may  increase  the 
difficulty  of  retaining  people  currently  in  the  service.  The  combined  impacts  of  changes  in  social 
structure  and  social  attitudes  on  recruitment  and  retention  may  increase  the  difficulty  of  manning 
America's  Army. 


92 


Social  Structures,  Social  Systems,  and  Social  Networks 


Contract  #:  W74V8H-05-K-0007 
Institution:  The  University  of  Maryland 


Contract  Dates:  11/01/2004  to  10/31/2008 
PI:  David  R.  Segal 

Co-PI:  Mady  Wechsler  Segal  &  Meyer 
Kestnbaum 


Problem(s)/Research  Question(s)  -  Can  social  network  theory  help  to  understand  and  fight 
insurgencies? 

Technical  Barrier(s)  -  Social  network  theory  is  not  highly  developed;  data  on  insurgencies  are 
not  readily  available. 

Significance/Impact  for  Basic  Research  -  This  attempted  application  of  social  network  theory 
can  contribute  to  the  development  of  the  theory  of  the  theory  and  of  methods  of  data  collection. 

Potential  Transitions:  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  Army  training  for  2 1  st  century  conflict; 

•  Development  of  doctrine  for  2 1  st  century  conflict; 

•  Assisting  decision-makers  in  understanding  2 1  st  century  conflict;  and 

•  The  general  understanding  of  the  structure  of  networks. 


Overview 


The  armed  forces  of  the  United  States  have  formal  and  publicly  known  organizational 
structures  and  are  trained  to  engage  in  combat  with  similarly  structured  forces  of  other  major 
nations.  This  was  the  nature  of  most  of  our  military  engagements  during  the  twentieth  century, 
such  as  the  two  World  Wars,  and  has  been  the  basis  for  most  of  our  combat  doctrine.  However, 
the  conflicts  in  which  we  have  been  engaged  more  recently  have  increasingly  involved 
adversaries  that  are  not  structured  as  we  and  our  allies  are,  and  our  major  twentieth  century 
adversaries  were.  They  are  increasingly  less  likely  to  be  acting  as  agents  of  nations,  and  they  go 
to  great  lengths  to  conceal  the  nature  of  their  organization.  The  purpose  of  this  research  is  to 
apply  a  perspective  known  as  social  network  theory  to  the  understanding  of  the  Baathist 
insurgency  in  Iraq  after  the  fall  of  the  Saddam  Hussein  regime  in  Operation  Iraqi  Freedom,  to 
evaluate  its  contribution  to  understanding  the  structure  of  the  insurgency,  and  to  identify  its 
potential  contributions  to  counter-insurgency  doctrine  and  to  military  operations  against 
insurgencies.  We  feel  that  this  knowledge  will  contribute  to  an  understanding  of  twenty-first 
century  warfare  and  contribute  to  training  and  doctrine  with  regard  to  these  conflicts. 

Research  Approach 

The  data  collection  phase  of  this  research  is  a  field  study  of  the  military  operations  that 
led  to  the  capture  of  Saddam  Hussein.  The  analysis  of  these  data  from  the  perspective  of  social 
network  theory  helps  us  understand  which  elements  of  the  theory  are  useful,  how  the  theory  can 
be  developed  further,  and  how  it  can  be  applied  in  other  field  settings. 
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Our  data  come  from  the  Intelligence  Preparation  of  the  Battlefield  (IPB)  by  the  4lh 
Infantry  Division,  operating  in  the  area  of  Tikrit,  Iraq,  in  2003,  and  engaged  in  the  attempt  to 
capture  Saddam  Hussein,  who  was  moving  around  the  country.  One  of  the  elements  of  the  IPB 
was  the  development  of  link  diagrams,  showing  anyone  known  to  be  related  to  Saddam  Hussein 
by  blood  or  tribe.  Additional  intelligence  information  identified  people  who  were  not  related  to 
Saddam  Hussein  in  these  ways,  but  were  functionally  related  to  him,  e.g.,  were  drivers, 
bodyguards,  or  secretaries.  Building  the  diagrams  involved  identifying  the  roles  that  individuals 
played,  according  to  the  Division's  understanding  of  organizational  functioning  (e.g.,  chief  of 
staff). 

Accomplishments 

The  basic  structure  of  Saddam  Hussein's  network  was  identified  in  the  link  diagrams. 
Ultimately,  214  interconnected  individuals  were  identified  in  the  network.  Figure  1  shows  the 
basic  link  diagram.  Note  the  basic  links;  the  dimensions  of  this  report  do  not  allow  for  a  full, 
readable  diagram.  The  basic  structure  of  the  network  was  very  simple.  Only  a  small  proportion 
of  the  total  possible  ties  between  individuals  actually  existed.  The  network  was  not  very  dense. 
Despite  the  fact  that  most  of  the  members  of  the  network  were  extended  or  immediate  family, 
most  people  were  directly  connected  to  very  few  others.  Thus,  they  were  not  in  a  position  to  give 
information  on  many  other  people  if  captured.  The  average  number  of  steps  necessary  to  get 
from  one  person  to  another  was  between  five  and  six.  The  largest  group  of  people  in  which  each 
was  connected  to  all  others  in  the  group  was  three,  and  there  were  only  thirteen  such  three  person 
cliques.  Family  relationships  account  for  most  of  these  three-person  cliques.  Only  twenty-three 
people  had  direct  ties  to  Saddam  Hussein,  and  as  Figure  2  shows,  none  of  these  people  had  direct 
ties  to  any  of  the  others.  Only  two  three-person  groups  are  linked  directly  to  Hussein.  Other 
members  of  the  network  were  connected  to  Hussein  indirectly,  through  one  or  more  of  these 
twenty-three  people.  Most  members  of  the  network  were  at  some  considerable  social  distance 
from  Saddam  Hussein.  Thus,  Saddam  Hussein  was  extremely  insulated.  A  journal  article 
reporting  these  findings  has  been  published. 
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Figure  I.  Full  network  of  Saddam  Hussein. 


Figure  2.  Saddam  Hussein's  sub-network. 

Contributions  to  Basic  Research 

The  most  dramatic  contribution  of  this  study  to  basic  research  might  be  the  confirmation 
of  population  structure  that  has  been  demonstrated  in  previous  and  very  different  studies  of 
social  network  structure.  Stanley  Milgram’s  “small  world”  experiments  in  the  1960s  suggested 
that  people  in  the  United  States  are  connected  by  six  links,  on  average.  More  recently,  Duncan 
Watts,  in  an  international  study,  found  that  the  average  number  of  intermediaries  needed  to  get 
an  electronic  communication  between  two  people  internationally  was  about  six.  While  neither  of 
these  studies  supports  the  notion  of  “six  degrees  of  separation”  made  popular  by  the  stage  play 
and  film  of  that  name  in  the  early  1990s,  by  the  first  season  Battlestar  Galactica  episode  with 
that  name,  or  the  recent  television  series,  Six  Degrees ,  they  all  suggest  that  about  five  points  are 
necessary  to  link  people,  on  average.  In  the  current  analysis,  as  Figure  3  shows,  more  than  half  of 
the  members  of  the  network  were  linked  to  each  other  by  five  or  fewer  intermediate  steps,  and 
about  two  thirds  of  the  people  in  the  network  were  linked  within  six  degrees  of  separation.  Thus, 
the  structure  of  this  insurgency  network,  while  less  dense,  was  not  very  different,  at  least  in 
terms  of  interconnectedness,  from  other  social  networks  that  have  been  analyzed.  With  nine 
degrees  of  separation,  80%  were  linked.  However,  in  Saddam  Hussein's  network,  20%  still 
needed  more  than  nine  intermediary  steps  to  be  linked  together.  This  finding  contributes  to  our 
understanding  of  network  structures. 
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Number  of  Paths  to  Saddam  Hussein 


Figure  3.  Number  of  paths  to  Saddam  Hussein. 

Equally  important,  it  demonstrates  that  when  organizational  structure  is  unknown,  and 
indeed  when  attempts  are  made  to  keep  it  from  being  known,  the  structure  may  nonetheless  be 
inferred  from  publicly  available  information,  such  as  newspapers,  communications  patterns  such 
as  e-mails  and  telephone  calls,  and  interrogations. 

Potential  Army/Military  Applications 

The  most  dramatic  application  is  that  Saddam  Hussein  was  captured.  Using  the  network 
structure,  the  Army  staged  a  series  of  raids  in  November  and  December  2003  that  first  caught  a 
number  of  mid-level  leaders,  then  the  inner  circle,  and  ultimately  Saddam  Hussein  himself.  As  a 
result  of  this  success,  an  appendix  on  social  network  analysis  has  been  included  in  the  Army's 
new  field  manual  on  insurgency.  Doctrine  on  social  network  analysis  is  being  developed  and 
Army  schools,  including  West  Point,  have  begun  teaching  techniques  of  social  network  analysis. 

Future  Plans 

During  the  next  year,  we  will  continue  analysis  of  these  network  data  to  evaluate  the 
utility  of  the  various  metrics  that  have  been  developed  for  understanding  the  structure  of  social 
networks. 
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Construct  Validating  Aspects  of  the  Theory  of  Successful  Intelligence 
via  a  Test  Battery  for  Measuring  Mental  Flexibility 
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Problem(s)/Research  Question(s)  -  Mental  flexibility — the  ability  to  cope  with  novelty  and  to 
establish  automatized  levels  of  information  processing — is  an  important  aspect  of  intelligence  as 
it  is  conceptualized  in  Sternberg's  triarchic  theory  of  successful  intelligence.  The  purpose  of  this 
project  is  to  create  a  new  assessment  instrument  that  simultaneously  provides  partial  construct 
validation  of  the  theory  of  successful  intelligence  (Sternberg,  1985)  and  a  means  for  assessing 
mental  flexibility. 

Technical  Barrier(s)  -  There  currently  exists  no  broadly  conceived  test  of  mental  flexibility, 
hence,  a  measure  is  needed  that  will  do  justice  to  the  construct. 

Significance/Impact  for  Basic  Research  -  Given  the  importance  of  mental  flexibility  in  a 
rapidly  changing  world,  and  the  fact  that  such  flexibility  is  not  currently  assessed  within  the 
framework  of  conventional  psychometric  tests,  such  a  test  seems  to  have  practical  utility  as  well 
as  theoretical  justification.  In  battlefield  situations,  circumstances  change  rapidly  and 
commanders  need  to  be  able  to  change  their  plans  quickly  and  effectively  in  response  to 
changing  conditions.  For  example,  the  shape  of  the  conflict  in  Iraq  changes  continually,  and 
effective  response  requires  flexibility  in  recognizing  the  changing  posture  of  the  enemy. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  SELECT2 1 :  Selection,  Classification  and  Performances  Metrics  for  the  Future  Force 
(WP  257)  and  Selection,  Classification,  and  Performance  Metrics  for  the  Future  Force 
Soldier  (ATO:  IV.HS.  2002.01); 

•  LEADERSHIP:  Developing  Leaders  in  a  Changing  Army  (WP  103); 

•  Accelerating  Leader  Development  (WP  269)  and  ATO  III. HS. 2004. 01 ; 

•  Future  Force  Warrior  Training  ( WP  2 1 5);  and 

•  FUTURE-TRAIN:  Techniques  and  Tools  for  C4ISR  Training  of  Future  Brigade  Combat 
Team  Commanders  and  Staffs  (WP  211). 


Overview 


Intelligence  tests  have  always  been  subject  to  heavy  criticism.  One  of  the  most  prominent 
criticisms  regards  the  lack  of  precision  in  predicting  success  in  dealing  with  real-life  problems, 
such  as  educational  or  occupational.  Thus,  it  can  be  questioned  whether  scores  on  traditional 
intelligence  tests,  taken  alone,  are  optimal  for  predicting  success  in  problem  solving  in  a  rapidly 
changing  world  or  even  whether  the  traditional  concept  of  "intelligence"  is  wholly  adequate  for 
this  purpose.  An  appropriate  assessment  procedure  should  reflect  a  person's  ability  to  adjust  in  a 
flexible  manner  to  changing  situational  requirements. 
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The  theory  of  successful  intelligence  specifies  not  only  the  kinds  of  broad  abilities 
(analytical,  creative,  and  practical)  that  play  a  role  in  achieving  success,  but  also  the  cognitive 
processes  required  to  apply  these  abilities  and  the  problem-solving  strategies  through  which 
success  may  be  achieved.  Within  the  framework  of  the  triarchic  theory  of  successful  intelligence, 
intelligence  is  conceptualized  on  three  different  levels.  On  the  first  level,  the  focus  is  primarily 
on  the  interaction  of  cognitive  components  as  the  elementary  and  universal  units  of  information 
processing  (componential  subtheory).  The  second  level  deals  with  the  relationship  of  intelligence 
to  the  experience  of  the  individual  (experiential  subtheory).  On  the  third  level,  intelligence  is 
defined  in  terms  of  the  context  in  which  intelligent  behavior  occurs  (contextual  subtheory).  We 
hypothesize  that  Mental  Flexibility  will  be  expressed  through  all  three  levels,  and  are  assessing 
Mental  Flexibility  in  the: 

•  componential  subtheory  through  tests  of  Flexible  Inference  and  Flexible  Mapping, 

•  experiential  subtheory  through  tests  of  Figural  and  Verbal  Counterfactual  Analogies,  and 

Insight  problems, 

•  contextual  subtheory  through  the  Flex  Art  test 

The  goal  of  this  project  is  to  develop  a  new  assessment  instrument  based  on  the  theory  of 
successful  intelligence  to  measure  mental  flexibility.  Mental  flexibility,  as  we  conceptualize  it,  is 
a  part  of  creative  thinking,  but  not  the  only  part.  Aspects  of  personality  (e.g.,  openness  to 
experience,  need  for  cognition)  and  motivation  (e.g.,  goal  orientation)  also  contribute  to  creative 
thinking.  As  a  sub-construct,  mental  flexibility  manifests  itself  at  every  level  of  the  theory  of 
successful  intelligence  (componential,  experiential,  and  contextual).  Accordingly,  we  have 
created  a  multifaceted  test  of  flexible  thinking  derived  from  the  theory  to  measure  how  well  one 
can  apply  the  components  of  intelligence  to  relatively  novel  tasks  and  situations.  Given  the 
importance  of  flexibility  in  a  rapidly  changing  world,  and  the  fact  that  mental  flexibility  is  not 
currently  assessed  within  the  framework  of  conventional  psychometric  tests  (Sternberg,  1981 ), 
such  a  test  seems  to  have  practical  utility  as  well  as  theoretical  justification. 

Research  Approach 

Broadly  speaking,  the  concept  of  successful  intelligence  recognizes  the  importance  of 
socio-cultural  context,  in  addition  to  multiple  personal  capabilities,  in  determining  success.  This 
conception  stands  in  contrast  to  traditional  views  of  intelligence,  which  posit  a  single  personal 
capability  that  determines  success  across  performance  situations.  Successful  intelligence  is  the 
balancing  of  analytical,  creative,  and  practical  abilities  to  achieve  success  within  a  particular 
socio-cultural  context.  In  other  words,  success  within  a  particular  socio-cultural  context  is 
determined  by  one’s  ability  to  capitalize  on  one’s  strengths  and  compensate  for  one's 
weaknesses  to  enact  strategies  for  achieving  success. 

Mental  flexibility,  as  we  conceptualize  it,  is  a  part  of  creative  thinking,  but  not  the  only 
part.  Aspects  of  personality  (e.g.,  openness  to  experience,  need  for  cognition)  and  motivation 
(e.g.,  goal  orientation)  also  contribute  to  creative  thinking.  As  a  sub-construct,  mental  flexibility 
manifests  itself  at  every  level  of  the  theory  of  successful  intelligence  (componential,  experiential, 
and  contextual).  Performance  on  a  test  of  mental  flexibility  should  also  provide  incremental 
prediction  of  success  beyond  that  obtained  with  conventional,  g-based  tests. 
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Five  new  mental  flexibility  assessment  instruments  were  developed  and  underwent 
formative  and  summative  evaluation.  An  initial  item  pool  for  each  mental  flexibility  test  was  first 
developed  and  reviewed.  Initial  tests  were  piloted  via  a  combination  of  paper-and-pencil  and 
computerized  on-site  administration  to  a  sample  of  college  undergraduates.  The  pilot  data  were 
analyzed  and  revised  accordingly.  A  revised  and  expanded  mental  flexibility  battery  and 
validation  measures  were  administered  to  a  larger  sample  of  college  undergraduates  and 
analyzed.  This  revised  and  expanded  assessment  battery  contained  the  following  five  newly 
developed  instruments: 

1 .  Flexible  Inference.  This  new  test  of  mental  flexibility  is  computer-administered  and  made 
up  of  classification  problems  designed  to  assess  the  ability  to  infer  relations  flexibly.  It 
contains  18  items,  in  the  figural,  verbal,  and  numerical  content  domains.  Each  item 
contains  an  item  prompt  and  a  set  of  four  domain-consistent  response  pair  options,  one  of 
which  must  be  linked  to  the  prompt  by  inferring  common  properties.  Each  item  triplet 
contains  the  same  stimuli.  The  three  parts  of  an  item  differ  in  the  arrangement  of 
elements  in  response  pairs  such  that  a  common  property  must  be  inferred  to  link  the 
prompt  to  the  correct  pair.  To  solve  an  item  part,  previously  inferred  relations  must  be 
inhibited  and  new  ones  identified.  The  inferred  relation  that  links  a  prompt  to  the  correct 
response  pair  option  is  classified  as  domain  typical  or  domain  atypical.  Domain-typical 
relations  are  based  on  properties  that  are  dominant  and  might  typically  be  considered  in 
the  domain  of  reference.  Domain-atypical  relations  are  based  on  properties  that  might  be 
secondary  and  would  less  often  be  considered  in  the  domain  of  reference. 

2.  Flexible  Mapping.  This  new  test  of  mental  flexibility  is  computer-administered  and  made 
up  of  21  analogy  problems  designed  to  assess  the  ability  to  map  inferred  rules  across 
figural,  verbal,  and  numerical  content  domains.  Each  item  triplet  is  made  up  of  an 
analogy  that  varies  in  terms  of  the  content  domain  to  which  it  must  be  mapped.  For  each 
triplet,  one  part  must  be  mapped  to  the  same  content  domain  (< domain  homogeneous ),  and 
two  must  be  mapped  to  different  content  domains  ( domain  heterogeneous). 

3.  Counterfactual  Analogies  (CFA).  This  is  a  set  of  two  tests  of  mental  flexibility  made  up 
of  counterfactual  (novel)  and  familiar  analogy  problems  drawn  from  Sternberg  ( 1 986) 
and  designed  to  assess  the  ability  to  cope  with  relative  novelty  in  the  verbal  and  figural 
domains.  Tests  contain  a  mix  of  items,  some  requiring  reasoning  based  on  facts  and 
others  requiring  reasoning  based  on  counterfactual  (novel)  premises. 

(a)  CFA- Verbal  is  a  computer-administered  test  that  contains  48  verbal  analogy 
items  with  four  response  options.  Applying  a  scheme  developed  by  Sternberg  and 
Gastel  (1989a,  1989b),  all  items  are  preceded  by  a  premise  that  is  either  familiar 
or  counterfactual  (novel),  and  either  relevant  or  irrelevant.  Participants  are  first 
presented  with  the  premise  and  given  as  long  as  they  wish  to  read  it.  They  then 
press  a  button,  which  results  in  the  disappearance  of  the  premise  and  the 
immediate  appearance  of  the  analogy  item.  Accuracy  scores  for  CFA-Verbal  are 
calculated  by  summing  correct  responses. 
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(b)  CFA-Figural  is  a  computer-administered  test  that  contains  30  figural  analogy 
items  with  four  response  options.  Applying  a  partially  modified  scheme  of  one 
developed  by  Sternberg  and  Gastel  (1989),  all  items  are  preceded  by  a  premise 
that  is  either  familiar  or  counterfactual  (novel).  Items  were  equally  divided  among 
these  two  premise  types.  Participants  are  first  presented  with  the  premise  and 
analogy  stem,  and  are  given  as  long  as  they  wish  to  view  it.  They  then  press  a 
button,  which  results  in  the  disappearance  of  the  premise  and  the  immediate 
appearance  of  the  analogy  item.  Accuracy  scores  for  CFA-Figural  are  calculated 
by  summing  correct  responses. 

4.  Insight  Test.  This  new  test  of  mental  flexibility  is  a  paper-and-pencil  administered  test  of 
coping  with  novelty  through  insight.  It  contains  nine  insight  problems  drawn  from  the 
literature  (Fixx,  1972;  Metcalfe,  1986a;  Seifert  &  Patalano,  1991;  Sternberg  &  Davidson, 
1982;  Weisberg,  1988)  that  represent  a  mix  of  verbal,  figural,  and  numerical  problem 
types.  Participants  are  asked  to  provide  open-ended  responses  to  insight  problems  and  are 
given  as  long  as  they  wish  to  complete  the  test.  A  sample  problem  reads  as  follows,  “A 
bottle  of  wine  costs  $  1 0.  The  wine  was  worth  $9  more  than  the  bottle.  How  much  was  the 
wine  worth?”  (Sternberg  &  Davidson,  1982). 

In  order  to  validate  the  newly  developed  assessments  described  above,  we  also 
administered: 

•  Two  tests  of  cognitive  ability:  (1 )  the  Berlin  Model  of  Intelligence  Structure  (BIS:  Jager, 
1982,  1984),  and  (2)  the  French  Kit  of  Factor-Referenced  Cognitive  Tests  (F-Kit: 
Ekstrom,  French,  Harmon,  1976).  The  BIS  is  a  bimodal  hierarchical  model  for  describing 
broad  intellectual  abilities  in  the  framework  of  four  operational  components  (processing 
speed,  memory,  creativity,  and  processing  capacity)  and  three  content-based  components 
(figural,  verbal,  numerical),  which  are  composed  of  12  facets  of  performance.  The  French 
kit  is  a  battery  made  up  of  a  set  of  72  marker  tests  for  23  cognitive  aptitude  factors.  Two 
subtests  (Letter  Sets  Test  -1-1  [rev.].  Locations  Test  -1-2)  of  three  that  comprise  the 
induction  factor  (convergent)  were  administered. 

•  Four  tests  of  pattern  recognition:  (1 )  Soluble/ Insoluble  Analogy  Test.  This  is  a  30-item 
multiple-choice  figural  analogy  test  developed  for  this  investigation.  It  is  designed  to 
measure  pattern  recognition  by  comparing  response  accuracy  on  soluble  versus  insoluble 
items;  (2)  Group  Embedded  Figures  Test  (GEFT;  Witkin,  Oltman,  Raskin,  &  Karp,  1971, 
2002).  This  test  is  an  adaptation  of  the  Embedded  Figures  Test  (EFT;  Witkin,  1950; 
Witkin,  Dyk,  Faterson,  Goodenough,  &  Karp,  1962)  modified  for  group  administration.  It 
measures  competence  in  perceptual  field  independence,  which  has  been  associated  with  a 
global-versus-analytical  dimension  of  cognitive  functioning;  (3)  Revised  Minnesota 
Paper  Form  Board  Test  (Likert  &  Quasha,  1970;  Paterson,  Elliott,  Anderson,  Toops,  & 
Heidbreder,  1930).  This  test  measures  the  capacity  to  visualize  and  manipulate  objects  in 
space;  and  (4)  Minnesota  Clerical  Test  (Andrew  &  Paterson,  1959).  This  is  a  test  of 
perceptual  speed  and  accuracy  in  recognizing  name  and  number  sequence  pairs. 


101 


•  Two  personality  tests:  (1 )  the  Cognitive  Flexibility  Scale  (Martin  &  Rubin,  1995).  This 
self-report  survey  measures  three  components  of  cognitive  flexibility  including:  (a) 
awareness  of  available  options  and  alternatives;  (b)  willingness  to  be  flexible  and  adapt  to 
situations,  and  (c)  self-efficacy  in  being  flexible;  and  (2)  the  NEO-Personality  Inventory 
Revised  (Costa  &  McCrae,  1992).  This  personality  survey  measures  five  dimensions 
including:  neuroticism,  extroversion,  openness,  agreeableness,  and  conscientiousness. 

Accomplishments 

We  completed  a  summative  evaluation  of  the  new  assessment  instruments  on  a  sample  of 
476  college  students.  The  newly  developed  mental  flexibility  tests  showed  adequate  reliability, 
and  preliminary  evidence  of  construct-  and  criterion-related  validity  as  measures  of  the  ability  to 
cope  with  novelty.  One  mental  flexibility  factor  explained  70%  of  variance  in  the  test  battery  and 
was  differentiated  from  the  latent  factor  underlying  divergent  and  convergent  measures  of  fluid 
intelligence.  Preliminary  evidence  of  incremental  criterion-related  validity  was  found,  suggesting 
that  the  mental  flexibility  test  battery  explains  variance  above  and  beyond  divergent  and 
convergent  measures  of  fluid  intelligence  in  criterion  measures.  Newly  developed  mental 
flexibility  tests  showed  a  consistent  and  strong  pattern  of  association  with  measures  of  pattern 
recognition,  suggesting  it  may  be  an  important  predictor  of  mental  flexibility. 

Contributions  to  Basic  Research 

According  to  the  theoretical  assumptions  within  the  framework  of  the  theory  of 
successful  intelligence,  mental  flexibility  reflects  the  ability  to  deal  with  novelty  and  to  establish 
automatized  levels  of  information  processing.  To  assess  this  ability  we  utilized  an  assessment 
approach  that  belongs  to  the  category  of  dynamic  testing.  As  has  been  shown  in  other  areas  (e.g., 
for  the  assessment  of  learning  ability,  see  Guthke  &  Beckmann,  2000,  2003;  see  also  Sternberg 
&  Grigorenko,  2002),  this  diagnostic  approach  represents  a  more  appropriate  way  to  assess 
intellectual  abilities  such  as  mental  flexibility.  In  contrast  to  traditional  approaches,  the  focus 
here  is  on  a  person's  ability  to  deal  with  standardized  variations  of  test  conditions.  In  addition  to 
this  methodological  improvement,  the  investigation  will  provide  further  insight  into  the  cognitive 
processes  underlying  intelligent  behavior  while  adapting  to,  shaping,  and  selecting  an 
environment  in  novel  situations.  Furthermore,  we  may  gain  another  piece  of  evidence  for  the 
validity  of  the  theory  of  successful  intelligence  with  its  broader  and  therefore  better  applicable 
conceptualization  of  intelligence. 

Potential  Army/Military  Applications 

Findings  suggest  that  the  newly  developed  test  battery  may  measure  flexible  cognitive 
ability  outside  the  framework  used  by  conventional  tests  of  fluid  intelligence,  supporting  the 
validity  of  the  experiential  subtheory  of  successful  intelligence.  The  mental  flexibility  test 
battery  developed  for  purposes  of  this  investigation  represents  an  initial  stage  in  the  development 
of  a  test  battery  that  could  potentially  be  used  for  selection  and  placement  in  educational  and 
occupational  settings.  Given  the  importance  of  mental  flexibility  in  a  rapidly  changing  world, 
and  the  fact  that  it  is  not  currently  assessed  within  the  framework  of  conventional  psychometric 
tests,  these  tests  seem  to  have  practical  utility  as  well  as  theoretical  justification. 
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Final  Summary 

The  goal  of  this  project  was  to  develop  and  evaluate  a  test  battery  that  assesses  mental 
flexibility  based  on  the  theory  of  successful  intelligence  (Sternberg,  1985).  Mental  flexibility  is 
defined  as  the  ability  to  cope  with  novelty  and  establish  automatized  levels  of  information 
processing.  This  research  simultaneously  provides  a  means  for  assessing  mental  flexibility  and 
validating  the  experiential  subtheory  of  the  theory  of  successful  intelligence. 

Five  new  mental  flexibility  assessment  instruments  were  developed  and  underwent 
formative  and  summative  evaluation.  An  initial  item  pool  for  each  mental  flexibility  test  was  first 
developed  and  reviewed.  Initial  tests  were  piloted  via  a  combination  of  paper-and-pencil  and 
computerized  on-site  administration  to  a  sample  of  college  undergraduates.  The  pilot  data  were 
analyzed  and  revised  accordingly.  A  revised  and  expanded  mental  flexibility  battery  and 
validation  measures  were  administered  to  a  larger  sample  of  college  undergraduates  and 
analyzed. 

The  newly  developed  mental  flexibility  tests  showed  adequate  reliability,  and  preliminary 
evidence  of  construct-  and  criterion-related  validity  as  measures  of  the  ability  to  cope  with 
novelty.  One  mental  flexibility  factor  explained  70%  of  variance  in  the  test  battery  and  was 
differentiated  from  the  latent  factor  underlying  divergent  and  convergent  measures  of  fluid 
intelligence.  Preliminary  evidence  of  incremental  criterion-related  validity  was  found,  suggesting 
that  the  mental  flexibility  test  battery  explains  variance  above  and  beyond  divergent  and 
convergent  measures  of  fluid  intelligence  in  criterion  measures.  Newly  developed  mental 
flexibility  tests  showed  a  consistent  and  strong  pattern  of  association  with  measures  of  pattern 
recognition,  suggesting  it  may  be  an  important  predictor  of  mental  flexibility. 

Findings  suggest  that  the  newly  developed  test  battery  may  measure  flexible  cognitive 
ability  outside  the  framework  used  by  conventional  tests  of  fluid  intelligence,  supporting  the 
validity  of  the  experiential  subtheory  of  successful  intelligence.  The  mental  flexibility  test 
battery  developed  for  purposes  of  this  study  represents  an  initial  stage  in  the  development  of  a 
test  battery  that  could  potentially  be  used  for  selection  and  placement  in  educational  and 
occupational  settings.  Given  the  importance  of  mental  flexibility  in  a  rapidly  changing  world, 
and  the  fact  that  it  is  not  currently  assessed  within  the  framework  of  conventional  psychometric 
tests,  these  tests  seem  to  have  practical  utility  as  well  as  theoretical  justification. 
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Problem(s)/Research  Question(s)  -  What  is  the  nature  of  the  structure  of  citizenship  and 
counterproductive  work  behavior  (collectively  known  as  discretionary  behavior)?  How  does 
discretionary  behavior  vary  over  time  within  individuals,  and  does  its  expression  vary  as  a 
function  of  changes  in  persons'  affective  (emotional)  states?  Also,  what  are  some  of  the  variables 
that  would  alter  the  relationship  between  emotions  and  discretionary  behavior  at  work  (e.g., 
leadership,  climates)? 

Technical  Barrier(s)  -  Models  of  how  discretionary  behavior  changes  within  individuals,  and 
the  effects  of  emotions  on  discretionary  behavior,  have  not  been  well  developed.  This  lacuna 
existed  in  large  part  because  the  methodological  and  statistical  tools  to  study  such  phenomena 
have  not  existed  until  recently. 

Significance/Impact  for  Basic  Research  -  Understanding  the  structure  of  discretionary 
behavior,  and  the  emotion-behavior  linkages,  will  lead  to  more  comprehensive  theory  and 
models  of  human  performance. 

Potential  Transitions  -  Knowledge  gained  in  this  basic  research  may  be  usefully  employed  in: 

•  WP  215  —  Objective  Force  Warrior  Training 

•  WP  269  —  Accelerating  Leader  Development; 

•  WP  283  —  Validating  Future  Force  Performance  Measures;  and 

•  WP  302  —  Ground  Systems  Team  Training. 

Overview 

In  spite  of  the  recent  interest  in  citizenship  and  counterproductive  work  behavior 
(collectively  called  discretionary  work  behavior),  and  their  documented  importance  to 
organizational  functioning,  much  remains  to  be  discovered  about  these  criteria.  Our  program  of 
research  integrates  the  burgeoning  literature  on  the  immediate  consequences  of  momentary 
affective  states  (see  Brief  &  Weiss,  2002,  for  a  review  of  this  literature)  with  the  literature  on 
discretionary  work  behavior.  We  are  conducting  several  studies  that  explore  and  map 
individuals'  naturally-occurring  on-the-job  citizenship  and  counterproductive  behavior,  the 
emotional  antecedents  of  such  behavior,  and  the  variables  that  potentially  influence  emotion- 
behavior  relationships  (e.g.,  leadership,  climates).  In  order  to  better  understand  the  emotional 
causes  of  citizenship  and  counterproductive  behavior,  we  first  have  to  understand  the  underlying 
dimensional  structures  of  this  behavior. 
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There  are  four  main  goals  of  our  project.  First,  we  are  examining  the  structure  of  work 
behavior  by  determining  their  co-occurrence  and  switching  over  time.  The  main  aims  of  this 
study  are  to  determine  which  behaviors  occur  together  on  a  given  time  occasion  (i.e.,  behavioral 
co-occurrence)  and  to  predict  changes  from  one  behavioral  state  on  one  occasion  to  the  next 
behavioral  state  on  a  subsequent  occasion  (i.e.,  behavioral  switching).  So  far,  the  literature  has 
not  examined  the  possibility  that  the  dynamic  structure  (i.e.,  using  “time”  as  the  level  of  analysis) 
of  citizenship  and  counterproductive  behavior  may  not  mirror  the  static  structure  (i.e.,  using 
“person”  as  the  level  of  analysis) — an  oversight  that  needs  to  be  rectified.  The  structure  of 
discretionary  behavior  also  needs  to  be  established  before  we  explore  its  emotional  antecedents. 

The  second  study  focuses  on  establishing  employees’  immediate  emotional  states  as  the 
causes  of  citizenship  and  counterproductive  work  behavior.  Research  examining  the  impact  of 
emotional  states  on  citizenship  behavior  and,  particularly,  counterproductive  behavior  is  rather 
meager.  Even  less  understood  are  the  work  events  that  instigate  the  immediate  emotional  states 
and  thereby  drive  the  display  of  this  behavior.  Although  the  basic  research  on  emotions  suggests 
that  momentary  affective  states  are  an  important  influence  on  both  citizenship  behavior  and 
counterproductive  behavior,  a  thorough  and  overarching  framework  for  studying  these 
relationships  in  detail  is  needed.  We  use  Affective  Events  Theory  (Weiss  &  Cropanzano,  1 996) 
as  a  unifying  framework  for  understanding  these  relationships  in  detail.  A  fundamental 
proposition  of  our  research  is  that  changes  in  the  likelihood  of  engaging  in  citizenship  and 
counterproductive  behavior  over  time  are  related  to  changes  in  a  person’s  affective  states. 
Therefore,  in  order  to  test  our  predictions,  we  will  examine  how  behavior  and  emotions  change 
within  people  over  time  rather  than  between  people.  This  approach,  known  as  a  within-person 
design,  will  allow  us  to  study  how  key  events  in  the  workplace,  and  the  way  supervisors  and 
organizations  create  and  manage  these  events,  influence  employees’  behavior  and  affective 
states. 


The  experience  of  negative  affective  states  does  not  always  lead  to  dysfunctional  work 
behavior;  similarly,  citizenship  behavior  does  not  always  follow  positive  affective  states. 

People’s  ability  to  “influence  which  emotions  they  have,  when  they  have  them,  and  how  they 
experience  and  express  those  emotions”  should  influence  whether  or  not  a  given  behavior  is 
exhibited  (Gross,  1999,  p.  557).  This  process,  known  as  emotional  regulation,  must  be  integrated 
into  the  study  of  the  way  emotional  states  instigate  counterproductive  behavior.  Individuals  vary 
in  their  ability  to  regulate  their  emotions,  but  situational  constraints  may  also  determine  whether 
individuals  have  the  ability  to  regulate  and  control  their  emotions.  It  is  likely  that  the  ability  to 
regulate  one’s  emotions  (or  lack  thereof)  will  influence  the  expression  of  counterproductive  and 
citizenship  behavior.  By  investigating  the  role  of  emotional  regulation,  researchers  will 
ultimately  be  able  to  develop  taxonomies  of  stable  regulatory  strategies  that  can  be  used  to 
control  the  dysfunctional  consequences  of  emotional  states. 

Employees’  emotions  and  behavior  occur  in  a  work  context  that  can  play  an  important 
role  in  facilitating  or  inhibiting  certain  behaviors  following  an  emotion-instigating  event. 
Therefore,  the  fourth  study  examines  leaders’  ability  to  regulate  their  subordinates'  behavior  via 
the  creation  of  “climates,”  leaders’  adeptness  at  identifying  others’  emotions,  and  their  skill  at 
managing  and  altering  these  emotions  in  an  effort  to  avert  counterproductive  behavior  on  the  part 
of  the  employee. 
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Research  Approach 

Contrary  to  traditional  cross-sectional  studies  that  look  at  differences  in  key  variables 
between  individuals,  we  adopt  a  within-person  approach  that  examines  how  variables  change 
within  an  individual  over  time.  Given  that  the  structure  of  discretionary  behavior  and  the 
associations  between  affect  and  such  behavior  are  likely  to  be  temporally  sensitive,  we  use  a 
dynamic  data  collection  tool  known  as  Ecological  Momentary  Assessment  (EMA).  Ecological 
momentary  assessment  has,  of  late,  begun  to  gain  popularity  in  the  organizational  psychology 
literature  (Weiss,  Nicholas,  &  Daus,  1999;  Alliger  &  Williams,  1993)  because  it  allows  for  the 
quantitative  analysis  of  work  behavior  in  its  natural  context  (Hormuth,  1986) — not  only  its 
natural  physical  context  but  also  its  natural  temporal  context.  EMA  can  therefore  be  used  to 
examine  issues  concerning  the  relatedness  of  variables  within  persons  over  time. 

In  order  to  capture  events  as  they  occur,  we  will  be  asking  participants  to  complete 
multiple  daily  questionnaires  on  Personal  Digital  Assistants  (PDAs),  such  as  Palm  Pilots™,  for 
data  collection.  Because  people  cannot  be  constantly  surveyed,  several  different  EMA  sampling 
strategies  have  been  developed.  Participants  can  be  surveyed  upon  the  occurrence  of  specified 
events,  or  a  fixed  or  random  amount  of  time  after  the  previous  survey.  The  durations  of  the 
studies  vary  as  a  function  of  the  particular  samples  used  and  the  research  questions  of  interest; 
however,  studies  will  typically  last  10-15  working  days.  Individuals  will  be  asked  to  provide 
reports  of  their  discretionary  behavior  and  affective  states  several  times  per  day. 

Accomplishments 

We  have  completed  data  collection  on  all  four  studies.  Study  1  was  presented  at  the  2006 
annual  meeting  of  the  Society  for  Industrial  and  Organizational  Psychology,  as  part  of  a 
symposium  on  innovative  methodological  research  on  citizenship  and  counterproductive  work 
behavior.  Subsequently,  Study  1  was  submitted  to  the  Academy  of  Management  Journal  (one  of 
the  premier  outlets  for  I/O  psychology  research).  We  received  a  “revise-and-resubmit."  and  are 
currently  in  the  process  of  revising  the  paper.  Study  2  and  the  pilot  data  from  Study  4  have  been 
submitted  for  consideration  at  the  2007  annual  meeting  of  the  Society  for  Industrial  and 
Organizational  Psychology.  Both  submissions  were  accepted.  We  are  currently  analyzing  the 
data  from  Study  3  and  have  just  completed  the  data  collection  on  Study  4.  We  also  have  one 
technical  report  under  review  at  ARI,  and  are  in  the  process  of  preparing  two  additional  reports. 

Contributions  to  Basic  Science 

Contrary  to  most  traditional  models  of  work  performance,  our  focus  on  employees" 
affective  states  as  predictors  of  performance  extends  the  existing  literature  by  providing  a 
framework  for  understanding  how  emotions  influence  discretionary  work  behavior  over  time.  In 
addition,  our  projects  integrate  theories  of  affective  experiences,  emotional  regulation,  and 
discretionary  work  behavior.  In  addition  to  these  theoretical  contributions,  our  findings  will 
provide  the  basis  for  developing  interventions  for  preventing  dysfunctional  work  behavior 
through  the  use  of  emotional  regulatory  strategies.  Employees  may  be  trained  to  recognize  the 
interplay  between  their  emotions  and  performance  at  work,  and  supervisors  and  organizations 


106 


can  develop  work  environments  that  would  foster  affective  states  that  lead  to  citizenship 
behavior  and  discourage  counterproductive  behavior  through  the  reduction  of  negative  affective 
states. 

Potential  Army/Military  Applications 

Negligent  and  willful  counterproductive/deviant  behaviors  are  especially  of  great 
importance  in  the  military,  where  the  cost  of  such  behavior  during  combat  operations  can  often 
be  measured  in  terms  of  lost  lives:  those  of  civilians,  of  one’s  fellow  Soldiers,  and — in  cases  of 
excessive  brutality  and  wanton  disregard  for  norms  of  civilized  combat — even  of  enemy 
combatants  and  detainees.  Consequently,  it  is  no  great  surprise  that  the  United  States  military  has 
long  recognized  the  importance  of  studying  counterproductive  (also  called  deviant  or  delinquent) 
behavior  among  its  own  Soldiers  (e.g.,  Bell  &  Holz,  1975;  Lennon,  1994).  Our  research  is 
unmistakably  relevant  to  the  U.S.  Army  in  its  quest  for  the  “good  Soldier.”  Clear  implications  for 
training  interventions  and  further  applied  research  in  Army  settings  can  be  drawn  from  this 
research. 

The  major  implication  of  our  research  is  that  behavior  and  emotions  are  transient  in 
nature.  They  may  therefore  have  a  dynamic,  temporal  structure  that  is  not  identical  to  their  static 
structure.  We  suggest  first  determining  the  temporal  structure  of  discretionary  behavior  in 
civilian  settings  with  their  inherent  lower-intensity  stressors,  and  then  conducting  more  applied 
research  with  the  military  to  examine  the  extent  to  which  previous  findings  hold  up  in  more 
extreme  situations — e.g.,  patterns  of  Soldier  behavior  under  (simulated)  enemy  fire  on  the 
battlefield.  This  subsequent  applied  military  research  can  take  the  form  not  only  of  analysis  of 
emotion-behavior  and  behavior-behavior  links  in  simulated  real-time  battlefield  environments, 
but  also  of  retrospective  investigations  into  egregious  incidents  committed  by  Soldiers  (e.g., 
torturing  prisoners,  executing  civilians,  shooting  their  own  colleagues,  committing  mutiny 
against  superior  officers,  etc.).  Such  research  would  provide  valuable  “red  flags”  that,  if  heeded, 
could  head  off  extremely  serious  incidents  before  they  actually  occur. 

Another  implication  of  our  proposed  research  is  that  emotional  responses  to  stressful 
and/or  aversive  events  can  be  regulated,  preventing  expression  of  counterproductive  behavior. 
Emotional  regulation  strategies  can  be  executed  either  by  the  Soldier  himself  or  herself,  or  by  the 
commanding  officer.  Research  on  emotion-behavior  links  and  emotional  regulation  are  perhaps 
even  more  important  in  the  military  than  in  other  settings,  given  research  suggesting  that  verbal 
behavior  in  the  military  has  ambiguous  emotional  nuances,  making  correct  interpretations  of  the 
speaker’s  intent  especially  difficult  (Wirshbo,  1990).  Officers  unable  to  correctly  discern  their 
subordinates’  emotional  states  are  unlikely  to  be  able  to  respond  with  appropriate  emotional 
regulation  strategies.  Future  research  in  military  settings  should  therefore  examine  situational 
and  individual  differences  in  emotional  judgment  related  to  oneself  or  others. 

Additionally,  our  research  can  lay  the  groundwork  for  the  study  of  officer-sponsored 
“climates”  that  create  contingencies  between  certain  behavior  and  certain  outcomes,  thereby 
attenuating  emotion-behavior  linkages.  For  instance,  an  officer  who  clearly  articulates  and 
consistently  enforces  a  policy  of  sanctions  for  poor  disciplinary  practices  may  be  able  to  reduce 
the  incidence  of  such  practices  regardless  of  the  frequency  and  magnitude  of  stressors  and 
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consequent  aversive  reactions  experienced  by  his  or  her  subordinates.  The  findings  of  our  studies 
are  likely  to  provide  material  relevant  to  courses  in  leader  development  and  training. 

Future  Plans 

We  received  a  "revise-and-resubmif '  on  a  paper  based  on  Study  1  that  we  submitted  to 
the  Academy  of  Management  Journal.  We  have  recently  resubmitted  this  paper,  and  are  awaiting 
the  journal's  response.  Moreover,  in  May  2007,  we  will  be  presenting  results  from  Study  2  and 
the  pilot  data  from  Study  4  at  the  Society  for  Industrial  and  Organizational  Psychology's  annual 
meeting.  Over  the  remainder  of  the  year,  we  plan  to  submit  these  findings  for  publication,  and 
also  to  complete  analyzing  and  writing  up  the  results  of  Study  3.  Moreover,  we  are  in  the  process 
of  preparing  our  second  and  third  technical  reports  for  ARI. 
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